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Incremental  development  entails  the  delib¬ 
erate  deferral  of  work  to  a  subsequent 
period,  using  technology  maturity  as  the 
measure  of  readiness.  This  article  illustrates 
that  this  approach  might  enable  more  effec¬ 
tive  delivery  of  the  first  increment  with  a 
comparison  of  two  major  systems  as  case 
studies.  But  there  are  some  inherent  risks  in 
an  evolutionary  approach  as  well,  and  the 
authors  caution  that  certain  attributes  of 
hardware  products  might  help  determine 
the  suitability  of  evolutionary  development 
methodologies.  Mutable  products  with  cost¬ 
less  production,  continuous  requirements, 
low  maintenance,  or  time  criticality  may  be 
more  likely  to  reap  advantages  from  evolu¬ 
tionary  approaches.  Products  that  are  nearly 
immutable,  have  binary  requirements  for 
key  capabilities,  require  man-rating,  or  are 
maintenance-intensive  may  not  be  best 
candidates  for  incremental  development. 
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It  Can  and  Cannot  Do 

Bill  Kobren 


Although  Performance  Based  Logistics 
(PBL)  sustainment  strategies  have  been 
used  successfully  for  the  last  decade, 
misperceptions  persist.  This  article  discusses 
what  PBL  is  and  is  not;  and  what  it  can  and 
cannot  do  for  the  military  services,  program 
managers,  and  ultimately  the  warfighter. 
PBL  is  not  about  contractors  on  the  battle¬ 
field  or  outsourcing  organic  workload.  It  is 
about  weapon  system  performance,  readi¬ 
ness,  best  value  outcomes,  capability,  and 
effective  and  efficient  warfighter  support. 
PBL  represents  a  fundamental  change  in 
how  DoD  supports  its  weapon  systems  and 
ensures  those  systems  are  reliable,  main¬ 
tainable,  and  available  when  and  where 
the  warfighter  needs  them.  When  it  comes 
to  delivering  performance  outcomes:  PBL 
works. 
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In  the  past,  war  planners  typically  treated 
acquisition  as  an  afterthought.  Operations 
Iraqi  Freedom  and  Enduring  Freedom 
Joint  Task  Force  commanders  were  not 
initially  afforded  the  full  value  of  acquisi¬ 
tion  capabilities  to  buy  local  resources  and 
manage  the  exploding  number  of  contrac¬ 
tors  in  the  Area  of  Responsibility  (AOR). 
Recognizing  this  shortfall,  DoD  created  the 
Joint  Contracting  Command  (JCC).  The  JCC 
provided  substantial  contracting  capacity 
and  coordination— critical  attributes  to  effec¬ 
tive,  efficient  AOR  operations.  This  research 
and  resultant  report,  originally  prepared  in 
2006  for  the  Industrial  College  of  the  Armed 
Forces,  played  a  substantial  role  in  shaping 
joint  thinking,  culminating  in  creating  Joint 
Publication  (JP)  4-10,  Operational  Contract 
Support.  JP  4-10  establishes  long-needed 
joint  contracting  doctrine  for  Combatant 
Command  AOR  operations. 
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Twenty-first  century  military  operations 
have  brought  forth  many  new  challenges 
for  the  Armed  Forces  of  the  United  States. 
One  such  challenge  is  with  new  operating 
environments,  where  current  systems  are 
not  always  effective.  While  it  is  desirable  to 
apply  a  systems  engineering  approach  to 
best  meet  critical  user  needs,  there  may  be 
a  misconception  that  systems  engineering 
requires  a  lengthy  and  detailed  process  not 
nimble  enough  for  a  rapid  prototyping  effort. 
This  article  describes  how  a  classic  systems 
engineering  methodology  was  successfully 
tailored  to  the  rapid  development  of  potential 
material  solutions  to  meet  a  critical  opera¬ 
tional  need.  Key  observations  are  drawn  from 
this  experience  and  formulated  into  heuristics 
for  tailoring  systems  engineering  for  future 
rapid  prototyping  efforts. 
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This  article  provides  a  brief  overview 
of  Foreign  Military  Sales  (FMS),  its  role 
in  the  ever  changing  dynamic  environ¬ 
ment  that  we  live  in,  and  finally  how 
some  of  the  specific  FMS  processes  can 
be  improved  through  the  application  of 
logistics  best  practices  and  initiatives.  The 
ultimate  goal  is  to  continually  improve  the 
processes  associated  with  FMS  to  create 
a  win-win  scenario  for  the  Department  of 
Defense  and  affected  stakeholders.  The 
best  practices  within  the  framework  of 
the  ten  integrated  logistics  support  (ILS) 
elements  discussed  within  this  article 
should  be  merged  into  the  current  acqui¬ 
sition  or  logistics  support  strategy. 


A  Multi-Criteria  Decision  Model 
for  Migrating  Legacy  System 
Architectures  into  Open  System  and 
System- Of -Systems  Architectures 

Cyrus  Azani 


Full-spectrum  dominance  in  battlefield, 
cost-effective  development  of  capabili¬ 
ties,  timely  reaction  to  evolving  threats 
and  technologies,  and  system  and 
process  flexibility  can  be  greatly  enabled 
through  the  application  of  open  systems 
strategies.  Such  strategies  are  effec¬ 
tive  business  and  technical  approaches 
for  assessing  the  appropriateness  of 
developing  modular  and  open  architec¬ 
tures  for  stand-alone  as  well  as  system 
of  systems.  This  article  will  introduce  an 
integrated  methodology  for  assessing 
existing  systems  and  migrating  their 
architectures  into  modular  and  open 
architectures.  The  proposed  method  inte¬ 
grates  open  systems  strategies  with  the 
Analytic  Hierarchy  Process  (AHP)  and 
Goal  Programming  (GP)—  two  powerful 
decision-making  models— to  evaluate  the 
appropriateness  of  open  systems  migra¬ 
tion,  rank  migration  candidates,  allocate 
resources  among  them,  and  develop  open 
architectures  for  selected  systems. 
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The  use  of  games  has  become  a  popular 
initiative  in  many  learning  organizations. 
The  Defense  Acquisition  University,  by 
targeting  enhanced  outcomes  for  learners 
and  using  innovative,  multiple  approaches 
to  develop  games  and  simulations  that  are 
engineered  to  yield  performance-oriented 
outcomes,  has  created  a  unique  oppor¬ 
tunity  to  reach  an  evolving  workforce  on 
multiple  levels.  Through  the  use  of  story 
and  interaction,  students  gain  a  better 
understanding  of  the  dynamic  application 
of  course  content,  while  fostering  moti¬ 
vation  to  learn  and  increasing  perceived 
relevance  of  the  instruction.  This  article 
covers  the  use  of  games  and  simulations 
in  three  different  initiatives:  Games  in 
Curriculum,  Games  in  Continuous  Learning 
Modules,  and  Mini-Games— each  of  which 
was  created  with  the  end  result  of  learning 
in  mind. 
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FROM  THE  EDITOR 


I  am  excited  to  announce  the  lineup  of  research 
articles  for  Issue  52  of  the  Defense  Acquisition  Review 
Journal.  The  first  article,  “From  Amorphous  to  Defined: 

Balancing  Risks  in  Evolutionary  Acquisition,”  by  COL 
John  T.  Dillard,  USA  (Ret.)  and  David  N.  Ford,  provides 
a  thought-provoking  analysis  of  Evolutionary  Acquisition 
(EA).  Their  goal  was  to  offer  meaningful  and  practical 
recommendations  to  program  managers  concerning  the 
effective  results  and  successful  implementation  of  EA. 

Spiral  development  is  a  management  strategy  designed 
to  reduce  risk,  but  the  nature  of  how  this  strategy  is  applied  can  introduce  additional  and 
different  risks.  The  authors  suggest  that  a  single  methodology  for  DoD  system  development 
may  not  be  appropriate,  and  they  provide  analysis  of  when  and  how  spiral  development 
can  be  used  most  effectively.  Stability  is  a  critical  characteristic  in  any  program— stability 
in  requirements,  design,  technology,  configuration,  and  most  of  all,  funding.  In  an  unstable 
world  with  an  uncertain  future,  the  only  constant  is  change.  This  article  will  help  program 
managers  cope  with  change. 

In  the  second  article,  “What  Performance  Based  Logistics  Is  and  What  It  Is  Not— And 
What  It  Can  and  Cannot  Do,”  author  Bill  Kobren  gives  our  readers  a  complete  description 
of  Performance  Based  Logistics  (PBL)  with  an  impressive  assessment  of  what  PBL  is  and 
what  it  is  not.  The  author  strongly  asserts  that  as  a  strategic  readiness  imperative,  PBL 
works!  Kobren  makes  the  case  that  PBL  delivers  substantial  improvements  in  performance 
with  lower  costs  across  the  life  cycle,  providing  more  for  the  warfighter  with  less  from 
the  taxpayer.  PBL  is  about  weapon  system  performance,  readiness,  best  value  outcomes, 
capability,  and  superior  support  to  the  warfighter.  PBL  represents  a  fundamental  change  in 
how  DoD  supports  its  weapon  systems  and  ensures  those  systems  are  reliable,  maintainable, 
and  available  in  the  most  cost-effective  manner. 

The  next  article,  “Joint  Acquisition  Command  Doctrine— A  Success  Story”  by  Al  Borzoo, 
Constance  S.  Short,  Ken  Brockway,  and  Col  Stan  L.  VanderWerf,  USAF,  provides  great 
insight  and  background  into  the  development  of  Joint  Publication  4-10,  Operational  Contract 
Support.  Prior  to  the  publication  of  JP  4-10  in  October  2008,  acquisition  had  been  mostly 
an  afterthought  in  war  planning.  As  a  result,  field  commanders  in  Operations  Iraqi  Freedom 
and  Enduring  Freedom  were  not  afforded  the  full  value  of  acquisition  capabilities  to  buy 
local  resources  and  manage  the  rapidly  growing  number  of  contractors  in  the  operations 
area.  In  today’s  dynamic  military  environment,  contractors  have  become  critical  to  Joint 
Task  Force  Operations.  As  operations  in  Iraq  were  to  prove,  insufficient  contractor  support 
planning  placed  great  strains  on  the  military’s  ability  to  manage  its  contractors.  While 
contracting  efforts  were  ultimately  successful,  insufficient  contract-management  capacity 
and  coordination  led  to  substantial  efficiency  losses  and  a  reduction  in  effectiveness.  JP 
4-10  now  addresses  these  issues  and  establishes  long-needed  joint  contracting  doctrine. 
Research  for  this  article  provided  many  recommendations,  which  were  used  in  the 
development  of  JP  4-10. 

In  the  fourth  article,  “Application  of  Systems  Engineering  to  Rapid  Prototyping  for  Close 
Air  Support,”  John  M.  Colombi  and  Richard  G.  Cobb  present  and  analyze  a  case  study  for 
successful  rapid  prototyping  without  compromising  sound  systems  engineering  principles. 
Twenty-first  century  military  operations  have  brought  forth  many  new  challenges  for  the 
U.S.  Armed  Forces.  One  such  challenge  is  with  new  unexpected  operating  environments, 
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From  the  editor 


where  current  systems  are  not  effective.  While  it  is  desirable  to  apply  a  systems  engineering 
approach  to  best  meet  critical  user  needs,  there  may  be  a  misconception  that  systems 
engineering  requires  a  lengthy  and  detailed  process  not  nimble  enough  for  a  rapid 
prototyping  effort.  This  research  article  describes  development  of  the  Friendly  Marking 
Device  (FMD)  that  allows  a  ground  controller  to  quickly  and  accurately  identify  the  position 
of  friendly  ground  personnel  to  close  air  support  aircraft.  The  authors  conclude  that  the 
FMD  project  successfully  applied  systems  engineering  principles  to  take  critical  user  needs 
and  rapidly  produce  viable  prototypes  that  could  be  quickly  transitioned  to  production.  Key 
observations  and  lessons  learned  are  discussed. 

The  following  article,  “Can  Applying  Organic  and  Industry  Best  Practices  Improve 
Foreign  Military  Sales  Supportability?”  by  Brian  B.  Yoo,  Duane  W.  Mallicoat,  and  Timothy 
"Tim"  K.  Simpson  delivers  a  comprehensive  look  at  the  history  of  the  Foreign  Military  Sales 
(FMS)  program  and  an  analysis  of  FMS  processes.  Several  specific  examples  of  FMS  cases 
are  presented,  and  several  Integrated  Logistics  Support  (ILS)  elements  are  covered  in  detail. 
The  FMS  program  is  authorized  by  the  Arms  Export  Control  Act  and  conducted  using  formal 
contracts  or  agreements  between  the  U.S.  Government  to  sell  weapon  systems  to  authorized 
foreign  purchasers.  The  activity  of  selling  weapon  systems  to  foreign  governments  becomes 
a  leveraging  tool  of  U.S.  foreign  policy  and  provides  the  United  States  an  avenue  to  conduct 
joint  operations  with  the  receiving  nation.  The  authors  make  the  case  that  FMS  is  a  critical 
tool  in  promoting  U.S.  foreign  policy  and  national  security  interests. 

The  sixth  article,  “A  Multi-Criteria  Decision  Model  for  Migrating  Legacy  System 
Architectures  into  Open  System  and  System-of-Systems  Architectures”  by  Cyrus  Azani, 
explores  the  application  of  the  Modular  Open  Systems  Approach  (MOSA).  Rapid  reaction  to 
evolving  threats  and  technology  requires  agile  system  architectures  that  could  quickly  and 
cost  effectively  be  integrated  and  reconfigured  within  family  of  systems  and  joint  system 
of  systems  warfighting  constructs.  Affordable  agility  and  reconfiguration  demands  open 
and  modular  forces,  systems,  and  system  of  systems.  Full-spectrum  dominance  on  the 
battlefield,  cost-effective  development  of  capabilities,  timely  reaction  to  evolving  threats 
and  technologies,  and  system  and  process  flexibility  can  be  greatly  enabled  through  the 
use  of  MOSA.  This  approach  is  an  effective  business  and  technical  strategy  for  assessing  the 
appropriateness  of  developing  modular  and  open  architectures  for  single  systems  as  well 
as  for  family  and  system  of  systems.  This  article  introduces  an  integrated  methodology  for 
assessing  the  migration  of  existing  system  architectures  into  modularand  open  architectures. 

The  final  article,  “Games  for  Good  — Flow  DAU  is  Using  Games  to  Enhance  Learning”  by 
Alicia  Sanchez,  is  the  second  in  our  new  series  of  technology  articles  from  DAU.  Sanchez 
summarizes  some  of  the  efforts  being  made  at  DAU  to  more  fully  integrate  Games  and 
Simulations  into  its  courses.  The  use  of  Games  and  Simulations  can  be  an  extremely  valuable 
learning  tool  enabling  increased  comprehension  and  retention.  Sanchez  is  leading  the  way  for 
increased  application  of  Games  and  Simulations  in  DAU  courses  and  other  learning  products. 

fM  Dr.  Paul  Alfieri 

(H  Executive  Editor 
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FROM  AMORPHOUS  TO 
DEFINED:  BALANCING 
RISKS  IN  EVOLUTIONARY 
ACQUISITION 

((  COL  John  T.  Dillard,  USA  (Ret.)  and  David  N.  Ford 

Incremental  development  entails  the  deliberate  deferral  of 
work  to  a  subsequent  period,  using  technology  maturity  as  the 
measure  of  readiness.  This  article  illustrates  that  this  approach 
might  enable  more  effective  delivery  of  the  first  increment 
with  a  comparison  of  two  major  systems  as  case  studies.  But 
there  are  some  inherent  risks  in  an  evolutionary  approach 
as  well,  and  the  authors  caution  that  certain  attributes  of 
hardware  products  might  help  determine  the  suitability  of 
evolutionary  development  methodologies.  Mutable  prod¬ 
ucts  with  costless  production,  continuous  requirements,  low 
maintenance,  or  time  criticality  may  be  more  likely  to  reap 
advantages  from  evolutionary  approaches.  Products  that  are 
nearly  immutable,  have  binary  requirements  for  key  capabili¬ 
ties,  require  man-rating,  or  are  maintenance-intensive  may 
not  be  best  candidates  for  incremental  development. 


Keywords:  Evolutionary  Acquisition,  Spiral 
Development,  Incremental  Product  Development, 
Risk,  Javelin,  ATACMS 
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Since  his  work  in  the  1830s,  Charles  Darwin  has  received  much  of  the 
credit  for  furthering  a  theory  of  biological  evolution.  While  not  the  first  to 
have  the  idea,  he  associated  observations  of  species  variety  on  the  island  of 
Galapagos  with  species  environment,  and  suggested  that  nature  selected  the 
variations  that  were  the  fittest  (Darwin,  1859).  In  its  time  (and  since),  the  idea 
was  considered  radical  and  a  threat  to  religious  and  social  order.  Mere  variety 
itself  can  be  controversial  since,  paradoxically,  variety  is  appreciated  in  some 
domains  (Ashby,  1960)  and  abhorred  in  others  (Neave,  2000). 

At  the  center  of  evolutionary  acquisition  are  also  ideas  and  phenomena 
about  variety  and  change.  As  a  policy  for  system  development,  it  too  is 
controversial.  And  as  within  Darwinian  concepts,  product  evolution  involves 
information  transfer,  interaction  with  the  environment,  and  unpredictability 
of  change  outcomes.  But  unlike  evolutionary  biology,  product  variations  and 
selections  occur  frequently  and  are  non-random.  Much  of  what  we  have  found  in 
the  following  research  on  evolutionary  development  and  project  management 
is  about  how  managers  must  cope  with  product  variety  and  change.  Using 
case  study  analyses,  review  of  current  subject  literature,  and  computational 
modeling  (expounded  upon  in  a  companion  article:  Ford  &  Dillard,  2009),  the 
focus  of  our  research  was  to  ascertain  the  program  management  implications  of 
evolutionary  acquisition,  obtain  lessons  learned  in  past  programs  as  applicable 
to  future  development  efforts,  model  and  simulate  projects  that  used  different 
acquisition  approaches,  derive  predictions,  and  make  recommendations  to 
project  managers  for  the  effective  and  efficient  harnessing  and  implementation 
of  evolutionary  acquisition. 


Background 

The  Department  of  Defense  (DoD)  promulgated  evolutionary  acquisition 
(EA)  as  policy  in  2000,  and  soon  after,  spiral  development  for  the  preferred 
acquisition  strategy  of  all  materiel  (Office  of  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics,  2000).  EA’s  goal  is  to  time-phase  system 
requirements  and  provide  capabilities  sooner.  But  confusion  over  terms  persists, 
despite  further  elaboration  and  even  codification  in  statute  (Armed  Forces,  2002), 
along  with  a  lack  of  full  understanding  of  many  policy  implications— especially 
some  inherent  risks.  DoD  policy  for  evolutionary  acquisition  mandated  spiral 
(i.e.,  amorphous  and  unplanned  requirements/technologies)  or  incremental  (i.e., 
defined  and  deferred  requirements/technologies)  development  methodologies 
for  all  programs.  Since  all  amorphous  spirals  eventually  become  defined 
increments,  the  disappearance  of  this  term  “spiral”  in  most  recent  (2008)  policy 
will  not  be  missed. 

Fundamentally,  E  A  means  there  will  always  be  multiple  product  releases  of  an 
item.  The  current  policy  thrust  is  primarily  about  the  reduction  of  product  cycle 
time  within  an  uncertain  environment  by  using  mature  technology  exclusively.  The 
DoD’s  requirements  process  has  also  followed  with  “evolutionary”  requirements 
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documents— a  new  idea.  Uncertainty  is  the  usual  realm  of  program  managers 
(PM),  especially  in  defense  systems,  and  is  usually  dealt  with  by  seeking  best 
information  (Pich,  Loch,  &  De  Meyer,  2002).  Earlier  reform  initiatives  were  aimed 
at  overcoming  information  gaps  and  technology  lag.  For  example,  the  1990s 
Integrated  Product  and  Process  Development  initiative  was  about  collaboration 
for  early  and  complete  requirements  realization.  However,  the  current  paradigm 
is  to  allow  uncertainty  in  requirements  to  resolve  over  time  and  endeavor 
only  what  is  immediately  achievable.  The  Government  Accountability  Office 
(GAO)  has  also  urged  the  DoD  to  move  toward  Knowledge-Based  Acquisition, 
with  Technology  Readiness  Levels  (TRL)  as  the  rubric  for  program  initiation 
(advanced  development)  (GAO,  1999a).  Thus,  at  the  very  heart  of  EA  is  the 
exclusive  use  of  mature  technology  to  reduce  project  scope. 


Observations  and  Assessments: 

Implications  of  the  Policy 

We  have  managed  and  observed  development  efforts  that  employed 
evolutionary  development  approaches  successfully.  Two  development 
programs  of  the  1990s— the  Army  Tactical  Missile  System  ( ATACMS )  and  the 
Javelin  anti-tank  missile  system— were  compared  herein  with  regard  to  their 
differing  acquisition  strategies,  TRLs,  and  program  outcomes.  Our  study  results 
support  that  a  more  effective  delivery  of  the  first  increment  might  be  facilitated 
with  an  evolutionary  approach.  However,  the  latest  EA  policy  implications 
and  outcomes  are  yet  to  be  fully  known,  and  some  authors  have  already 
expressed  insightful  strategic  and  institutional  oversight  concerns  (Sylvester 
&  Ferrara,  2003).  We  have  also  described  operational  program  management 
concerns  about  its  implementation,  including  excessive  decision  bureaucracy, 
organizational  challenges  from  multiple  and  concurrent  development  efforts, 
outmoded  technology  at  release,  funds  forecasting,  transaction  costs,  and 
maintenance  of  subsequent  increment  priority  (Dillard,  2005).  Our  additional 
findings  suggest  that  incremental  development  may  not  be  appropriate  as  a 
one-size-fits-all  strategy. 


VARIETY  AND  COMPLEXITY 

For  example,  variety  and  complexity  are  elements  of  project  risk.  While 
variety  and  product  diversity  are  preferred  by  market  consumers  to  satisfy 
mainstream  and  smaller  niche  needs,  variety  adds  complexity  in  production 
and  is  costly  for  hardware  manufacturers  and  owners  alike.  In  support  of  EA 
policy,  the  GAO  has  often  used  product  examples  such  as  home  appliances  and 
commercial  vehicles,  which  tend  to  ignore  product  variety  from  the  vantage 
point  of  fleet  owner  vs.  that  of  the  producer  (GAO,  1998a,  1998b,  1999b, 
2003).  The  DoD  is  unique  in  that  it  almost  entirely  outsources  capital  projects 
for  exclusively  internal  use,  and  this  aspect  of  lifelong  ownership  of  an  entire 
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fleet  of  systems  presents  a  different  relationship  than,  for  example,  a  product 
manufacturer  may  have  with  its  production  aircraft. 

Traditional  views  about  variety  from  late  design  changes  are  usually  negative, 
except  for  producibility  savings  and  performance  enhancements.  Changing 
production  configurations  is  not  viewed  as  efficient  due  to  supportability  issues 
(regarding  spares  and  maintenance)  with  lot,  model,  and  type  diversity.  RAND’s 
study  of  support  considerations  for  the  current  mixed  configuration  fleet  of 
Unmanned  Aerial  Vehicles  (UAV)  reported,  “Multiple  aircraft  configurations 
drive  multiple  spare  component  packages  and,  in  the  most  extreme  cases,  may 
drive  multiple  pieces  of  test  equipment,  all  significantly  increasing  long-term 
support  costs”  (Drew,  Shaver,  Lynch,  Amouzegar,  &  Snyder,  2005;  emphasis 
added).  Reliability  issues  can  also  emerge  because  of  insufficient  testing  of 
the  changes.  Depending  on  the  degree  of  change,  system  validation  and 
qualification  become  a  concern  if  changes  are  not  under  strict  control.  There 
may  be  backward  compatibility  and  interoperability  issues  as  well.  Another 
burden  is  the  training  impact  of  mixed  capabilities  within  the  force  or  even 
within  the  same  owning  and  operating  unit. 

Higher  levels  of  risk  from  system  complexity  are  generally  believed  to  be 
mitigated  by  control  measures,  as  within  organizational  contingency  theory  (i.e., 
centralization/decentralization,  etc.).  The  American  nuclear  Navy  was  rooted  in 
CAPT  Hyman  G.  Rickover’s  visit  to  Oak  Ridge  National  Laboratory  in  1946  to 
investigate  the  feasibility  of  using  nuclear  power  aboard  submarines.  During 
his  long  tenure  as  head  of  the  nuclear  program,  he  maintained  fundamental 
principles  about  technical  and  organizational  program  structures,  not  the  least 
of  which  was  personal  accountability.  Those  who  have  worked  with  acquisition 
of  nuclear  plant  materials  know  well  both  the  specifications  and  standards 
of  quality  that  are  unique  to  this  commodity,  as  well  as  Rickover’s  tenets  of 
responsibility  and  accountability  that  are  still  in  place.  They  are  largely  believed 
to  be  important  aspects  of  how  he  successfully  dealt  with  the  complexities 
and  uncertainties  of  a  new  application  of  technology.  The  Guide  to  the  Project 
Management  Body  of  Knowledge  (PMBOK)  (Project  Management  Institute, 
2004)  also  asserts  that  change  in  the  course  of  projects  and  products  is 
inevitable  and  mandates  the  need  for  a  disciplined  change-control  process  to 
control  its  impacts— from  inception  to  completion.  Many  other  useful  theorems 
on  systems  complexity,  change,  and  control  exist  to  alleviate  unwanted  variation 
in  development  and  production. 


CYCLE-TIME  AND  PHASE  CONCURRENCY 

We  have  observed  that,  though  concurrency  is  a  necessary  ingredient  for 
efficient  project  management,  it  has  also  long  been  correlated  with  risk  (due 
to  interdependence  of  activities)  and  might  vary  significantly  with  the  types  of 
activities  underway— inferring  that  periods  of  stable  production  configuration 
between  development  increments  reduce  complexity  in  program  structure  and 
attendant  risks.  Cycle-time  for  the  development  of  each  increment,  and  the 
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relatively  successive  or  concurrent  phasing  of  the  follow-on  increments,  will  also 
have  a  definite  impact  on  program  structure,  budgeting,  project  complexity,  and 
organizational  issues,  etc.  For  reasons  that  we  have  brought  forth  in  our  work 
on  the  computational  modeling  of  evolutionary  development,  we  have  concerns 
about  the  conduct  of  incremental  development  programs  with  continual  and 
highly  concurrent  phasing  of  development  increments. 


The  more  projects  that  specialists  support,  the 

LESS  THEY  ARE  PROPORTIONATELY  AVAILABLE  TO  THE 
PROJECTS  DUE  TO  "QUEUING  INEFFICIENCIES.” 


Particularly  in  matrix  organization  structures,  as  is  often  the  case  with 
projects,  there  can  be  a  tendency  to  staff  multiple  projects  with  a  single 
specialist.  The  more  projects  that  specialists  support,  the  less  they  are 
proportionately  available  to  the  projects  due  to  “queuing  inefficiencies.”  Their 
availability  decreases  because  of  the  need  for  transition  between  projects 
(physical,  mental,  learning  curve,  etc.).  This  has  at  times  resulted  in  large  delays 
in  project  completion  (Smith  &  Reinhartsen,  1998).  Similarly,  Ibrahim  (2005)  has 
shown  that  discontinuous  enterprise  membership  is  a  contributing  factor  toward 
knowledge  loss  in  organizations  involved  in  large  complex  product  development 
processes.  Examining  knowledge  flows  across  product  life  cycles  reveals  that 
members  often  are  not  engaged  in  all  phases.  Whether  from  rotation  of  duties 
or  multi-tasking,  a  discontinuous  member’s  inaccurate  knowledge  could  cause 
a  functional  error  at  the  individual  level,  which  is  not  immediately  obvious  at 
the  enterprise’s  overall  project  level.  Ibrahim’s  findings  support  observations  of 
knowledge  loss  continuing  despite  investments  in  information  technology  and 
knowledge  management. 


Development  Case  Studies 

One  of  the  most  recent  monographs  we  found  on  emerging  results  of 
evolutionary  acquisition  is  by  RAND— on  five  immature,  non-man-rated  space 
systems.  Space  systems  are  somewhat  different  than  general  force  defense 
projects  (in  their  quantities  produced,  their  operational  space  environment, 
greater  proportional  front-end  investment,  and  technology  development 
periods).  RAND  also  found  that  evolutionary  policy  confusion  persists  and  that  EA 
added  program  complexity  and  uncertainty,  especially  with  regard  to  budgeting. 
Extending  their  findings  to  non-space  DoD  programs,  RAND  highlighted  the  EA 
challenges  of  programmatic  flux  (Drew,  Shaver,  Lynch,  Mahyar,  &  Snyder,  2005). 
They  feel,  and  we  agree,  that  EA  presents  the  opportunity  for  typical  non-space 
project  management  challenges  to  be  even  more  formidable. 
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For  such  traditional  defense  systems,  as  expository  cases  of  evolutionary 
acquisition,  we  analyzed  two  tactical  missile  programs  that  illustrate  both 
planned  and  unplanned  change:  the  Army  Tactical  Missile  System  (ATACMS) 
and  the  Javelin  Anti-Tank  Weapon  System  (Dillard  &  Ford,  2007).  Both  of  these 
systems  began  as  Defense  Advanced  Research  Projects  Agency  (DARPA) 
programs  in  the  1980s  and  were  fielded  to  forces  and  employed  in  combat  in 
the  1990/2000S.  See  the  full  report  at  http://www.acquisitionresearch. 
net/_files/FY2007/NPS-AM-07-002.pdf  for  a  detailed  description  of  these 
case  studies  and  our  use  of  them  to  investigate  evolutionary  acquisition  with 
computational  modeling. 

ATACMS  employed  both  incremental  and  spiral  strategies  for  product 
development,  benefiting  from  an  elegant,  modular  independent  architecture. 
This  program  was  able  to  omit  its  technology  development  phase  by  employing 
mature  technologies  for  a  leap-ahead  capability  in  range.  The  basic  system 
arrived  essentially  on  budget  and  schedule,  with  several  successive  variants, 
both  pre-planned  and  unplanned.  Years  later,  one  instance  of  a  minor  production 
change  (uncontrolled  variety)  caused  missile  failure  and  a  costly  refit  of  already- 
produced  missiles. 

In  contrast,  Javelin  used  the  single-step-to-full-capability  approach  for 
product  development.  With  much  greater  modular  interdependence,  the 
program  embarked  upon  advanced  development  with  immature  technologies 
in  several  critical  areas— causing  significant  cost  and  schedule  overruns.  The 
system  has  also  experienced  subsequent  design  changes  and  product  variety, 
but  they  have  consisted  more  as  running  production  changes  than  as  product 
variants  or  blocks. 

A  comparison  of  the  development  and  use  of  technology  in  the  ATACMS  and 
Javelin  projects  clearly  illustrates  the  impacts  of  technology  maturity  on  first 
increment  project  performance.  The  Table  compares  the  technology  maturity 
in  the  ATACMS  and  Javelin  projects  by  identifying  the  Critical  Technologies  for 
seven  subsystem  categories  that  both  products  employed.  For  each  subsystem, 
the  Technology  Readiness  Level  of  the  critical  technology  used  at  the  time  of 
insertion  into  the  development  is  shown.  The  ATACMS  project  used  only  critical 
technologies  with  a  minimum  TRL  of  6  and  an  average  of  8.1.  In  sharp  contrast, 
the  Javelin  project  used  technologies  with  a  maximum  maturity  of  TRL6  and 
an  average  of  TRL5.  The  ATACMS  project  used  significantly  more  mature 
technology  than  the  Javelin  project  and  reaped  the  rewards  of  program  success. 

The  relative  cost  and  schedule  performance  of  the  ATACMS  and  Javelin 
projects  reflects  the  differences  in  the  use  of  technology.  The  ATACMS  project 
had  no  Advanced  Development  Phase  Contract  Cost  Growth  and  only  6  percent 
schedule  growth  in  the  Advanced  Development  Phase.  But  the  Javelin  project 
experienced  over  150  percent  cost  growth  and  50  percent  schedule  growth 
in  Advanced  Development.  The  poorer  project  performance  when  less-than- 
mature  technology  was  used  supports  the  potential  effectiveness  of  EA  in 
managing  technology  risk. 
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TABLE.  COMPARISON  OF  PROGRAMS  USING  DIFFERENT 
DEVELOPMENT  APPROACHES  AND  TECHNOLOGY 
READINESS  LEVELS 

Key  Program  Characteristics  -  First  Increment  of  Capability 


Program 

Aspects 

ATACMS 

(Evolutionary) 

Javelin 

(Single-Step) 

DARPA 

Predecessor 

Assault  Breaker  1977-82 

Tank  Breaker  1981-82 

Ultimate 

Capability 

"Deep  Attack’’ 

“Fire  and  Forget" 

Subsystem 

Critical 

Technology 

TRL 

Critical 

Technology 

TRL 

Munition 

Lance  M74 

Bomblet 

9 

Tandem  Shaped 
Charges 

5 

Propulsion 

Solid  Rocket  Motor 

9 

Two-Staged  Solid 
Rocket  Motor 

5 

Flight  Control 

Fin  Surfaces 

9 

Fin  +  Thrust  Vector 
Control  Vanes 

6 

Guidance  and 
Control 

Inertial 

9 

Tracker  Software 
Algorithm 

4 

Safe/Arm  Fusing 

Mechanical 

7 

Electronic 

4 

Software 

Function  (Target 
Acquisition,  Fire 
Control,  etc.) 

Various 

6 

Various 

6 

Sensor 

N/A 

- 

Focal  Plane  Array 

5 

Cost  of  ~$700M 

Development 

~$700M 

Contract  Type  Fixed  Price 

Cost  Reimbursable 

Tech  Development  Phase 

0  Months 

27  Months 

Advanced  Development  Phase— Planned 

48  Months 

36  Months 

Advanced  Development  Phase— Actual 

51  Months 

54  Months 

Total  Time  in  Development 

51  Months 

81  Months 

Advanced  Development  Phase 

Schedule  Growth 

6% 

50% 

Advanced  Development  Phase 

Contract  Cost  Growth 

0% 

150% 
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Synthesis  of  these  cases  reveals  that  as  an  approach  oriented  primarily  on  the 
reduction  of  product  cycle-time,  evolutionary  development  is  highly  facilitated 
by  the  leveraging  of  mature  technologies.  Also,  system  mutability,  along  with 
other  factors  discussed  in  the  next  section— such  as  time  criticality  (user  risk) 
and  modular  interdependency— can  bolster  incremental  development  suitability. 
For  ATACMS,  an  “open,”  or  at  least  elegant,  architecture  was  fundamental 
for  modular  variety,  and  thorough  design  specification  and  configuration 
management  accountability  proved  essential  for  managing  the  complexity  of 
multiple  product  releases.  In  the  case  of  the  Javelin,  key  capabilities  depended 
upon  immature  technologies  and  at  least  one  binary  requirement,  to  the 
detriment  of  project  cost  and  schedule  outcomes.  In  stark  contrast,  modular 
interdependence  was  manifested  by  an  almost  total  system  redesign  for  lengthy 
and  costly  weight  reductions. 


Do  Product  Attributes  Affect  Evolutionary 
Applicability  and  Outcomes? 

More  questions  about  EA  include  whether  products  with  different  attributes 
(e.g.,  hardware  vs.  software,  buildings  vs.  electronics)  may  lend  themselves  more 
or  less  to  the  use  of  an  incremental  development  approach.  From  the  literature 
and  cases  we  examined,  we  offer  the  following  other  product  attributes  for  PMs’ 
careful  consideration  when  planning  product  capability  increments. 


MUTABILITY 

Perhaps  our  foremost  reservation  is  the  appropriateness  of  the  evolutionary 
development  process  for  all  project  sizes  and  product  commodities  in  toto,  and 
the  application  of  the  spiral  process  to  hardware  products  vs.  Dr.  Barry  Boehm’s 
(1985)  original  and  most  relevant  application  of  this  development  approach 
toward  software.  Boehm  himself  warned  of  “hazardously  distinct”  spiral  model 
imitations,  and  in  his  own  words  described  his  vision  of  the  spiral  process: 

The  spiral  development  model  is  a  risk-driven  process  model  generator. 
It  is  used  to  guide  multi-stakeholder  concurrent  engineering  of 
software-intensive  systems.  It  has  two  main  distinguishing  features. 
One  is  a  cyclic  approach  for  incrementally  growing  a  system’s  degree 
of  definition  and  implementation  while  decreasing  its  degree  of  risk. 
The  other  is  a  set  of  anchor  point  milestones  for  ensuring  stakeholder 
commitment  to  feasible  and  mutually  satisfactory  system  solutions 
(Boehm  &  Hansen,  2000,  p.  3). 

Clearly,  the  conceiver  of  this  spiral  notion  was  oriented  upon  amorphous 
requirements  and  continuous  stakeholder  inputs  for  the  alleviation  of  project 
risk  with  a  very  mutable  product  (Boehm,  1988).  The  nature  of  software  being 
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in  the  digital  rather  than  physical  realm,  it  is  particularly  conducive  to  rapid  and 
successive  revision  and  nearly  costless  production.  And,  Boehm  encourages 
varying  from  the  spiral  model  as  needed  and  reverting  to  a  sequential  model  if 
requirements  are  well  established  and  the  project  less  risky. 

Multiple  product  increments  do  not  often  appear  in  large,  static,  singular 
projects  such  as  bridges,  highways,  office  buildings,  or  in  other  project  areas 
that  have  typically  long  lead  times  or  product  cycles,  such  as  feature-length 
films,  pharmaceuticals,  etc.  These  are  what  we  call  nearly  immutable  products 
and  are  much  different  than  smaller  projects  (like  rapid  software  application 
development)  with  much  shorter  development  periods.  However,  as  with  almost 
everything  engineered  that  we  can  observe  in  the  physical  world,  even  these 
things  can  evolve  and  change  with  additions,  spin-offs,  sequels  (and  prequels), 
expansions,  etc.  Mutability  simplifies  change,  and  that  idea  can  be  extended  to 
many  DoD  projects. 


USER  RISK/SAFETY 

For  DoD,  product  attributes  that  are  aligned  with  Boehm’s  notion  of  process 
models  being  driven  by  risk  are  those  of  mission  or  time  criticality,  survivability, 
and  user  safety.  System  safety  is  often  described  in  terms  of  “man-rating”  as 
approval  for  safe  usage. 

Time-critical  or  enhanced  survivability  systems.  DoD’s  products  have  expanded  risk 
considerations  beyond  Boehm’s  models  of  commercial  software.  Extending  the 
idea  of  project  risk-as-a-driver  down  to  the  level  of  the  end  user,  it  might  seem 
logical  to  assume  that  time  criticality  of  the  need  or  mission  (in  which  risk  of 
not  achieving  project  success  actually  endangers  customer  lives),  might  be  a 
significant  factor  in  the  appropriate  application  of  the  evolutionary  process 
for  reduced  initial  product  cycle-time.  Perhaps  defensive  systems  are  a  good 
example.  The  immediate  need  for  a  Rocket-Propelled  Grenade  defeater  or 
an  Improvised  Explosive  Device  neutralizer  for  currently  deployed  forces  in 
Iraq  and  Afghanistan,  for  example,  clearly  dictates  that  lives  will  be  lost  if  a 
near-term  capability  is  not  achieved.  We  also  cite  as  an  example  the  National 
Missile  Defense  initiative  in  which,  given  the  view  of  near-term  threats,  early 
deployment  of  even  rudimentary  capability  has  been  deemed  preferable 
to  waiting  for  full  capability.  Such  urgency  likely  precludes  full  and  certain 
requirements  specificity. 

Non-man-rated  and  man-rated  systems.  In  the  same  vein,  non-man-rated  systems 
(i.e.,  UAVs  or  cave-exploring  robots— capabilities  in  which  operator  lives  are  not 
at  risk  if  the  product  fails)— may  also  lend  themselves  readily  to  rapid  innovation 
and  riskless  experimentation  cycles.  However,  user  hazard  levels  for  man-rated 
systems  may  be  an  entirely  different  matter. 
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Man-rated  systems  present  a  different  challenge.  Configuration  variety 
adds  technical  complexity  with  sometimes  unpredictable  interactions.  In  such 
projects  as  pharmaceuticals,  aviation,  vehicular  transportation,  etc.,  producers 
mitigate  safety  risks  with  thorough  analyses,  documentation  reviews,  validation 
testing,  and  other  control  and  verification  processes.  By  their  very  nature— with 
lethal  hazards  for  the  end  user  and  typically  lengthy  approval  criteria— these 
may  not  be  good  candidates  for  an  evolutionary  approach. 


LOGISTICAL  SUPPORT  PLANNED  DURING  SERVICE/SHELF  LIFE 

Our  observations  warn  that  multiple  configurations  of  hardware  products 
come  at  a  cost  for  fleet  ownership.  Veterans  of  new  system  deployments  across 
the  force/fleet,  or  throughout  any  large  using  organization,  know  the  difficulties 
of  rolling  out  a  configuration  change.  Benefits  of  standardization  have  long 
been  offered  via  production  economies  of  scale,  commonality  of  parts  across 
platforms,  and  interoperability.  If  the  ultimate  goal  is  to  have  standardization 
across  the  DoD’s  force,  owning  multiple  configurations  (variety)  of  a  system 
seems  in  opposition,  with  added  complexity  in  training  and  supply  support  of 
the  item.  The  logistical  maintenance  strategy  cannot  be  ignored— whether  the 
end-item  is  maintenance-intensive,  such  as  tactical  vehicles,  or  maintenance- 
free,  such  as  with  many  electronics  items  and  munitions,  and  situations  in  which 
physical  changes  are  completely  transparent  to  the  user.  For  multiple-product 
configurations,  the  acquisition  approach  could  have  a  huge  effect  on  the  total 
costs  of  ownership,  as  previously  mentioned  by  Drew  et  al.  (2005)  in  regard  to 
UAVs. 


RANGE  OF  REQUIREMENT  ATTAINMENT 

Most  requirements  are  “continuous,”  i.e.,  may  be  satisfied  in  varying  amounts 
of  attainment.  Thus,  ranges  of  their  satisfaction  can  be  flexibly  specified,  allowing 
for  thresholds  (minimum  values  of  attainment)  and  objectives  (optimal  values 
of  attainment).  Examples  are  range,  accuracy,  weight,  reliability,  etc.  However, 
we  have  found  that  some  requirements,  often  critical  ones,  are  more  binary  in 
nature  than  continuous.  They  have  a  much  narrower  range  of  attainment,  such 
that  they  are  essentially  pass/fail  or  go/no-go  in  their  demonstration.  Examples 
are  Windows-compatibility,  “soft”  missile  launch,  network  security,  physical  fit, 
leak-proof,  shock-/vibration-/drop-proof,  survivability,  horizontal-to-vertical 
flight  transition,  etc.  If  one  of  these  more  binary-type  requirements  happens  to 
be  a  Key  Performance  Parameter,  its  attainment  will  be  on  the  project’s  critical 
path  and  highly  dependent  upon  technical  maturity.  As  such,  it  might  practically 
dictate  the  length  of  the  entire  advanced  development  effort  and  make  division 
into  capability  increments  less  beneficial  as  a  development  strategy.  Though 
somewhat  correlated  with  product  reliability,  these  kinds  of  requirements 
demand  a  system  that  “either  works  or  it  doesn’t”  without  the  flexibility  afforded 
by  objectives  and  thresholds. 
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AMOUNT  OF  CHANGE- AND  THE  LURE  OF  MODULARITY 

We  subscribe  to  the  current  systems  theorists’  view  that  complexity  is 
comprised  of  numbers  (of  components),  connections  (interdependencies) 
and  distinctions  (variety).  Distinction  corresponds  to  variety,  to  heterogeneity, 
and  to  the  fact  that  different  parts  of  complex  systems  behave  differently 
(Heylighen,  1997).  Variety  is  a  component  of  Nobel  Prize  winner  Herbert  Simon’s 
explanation  of  complexity— many  different  parts  with  many  interactions.  Simon 
argued,  from  his  observation  of  complexity  in  things  both  natural  and  artificial, 
that  complex  systems  evolve  from  simple  systems.  And,  they  do  so  more  rapidly 
when  there  are  stable,  intermediate  forms  or  sub-systems  (like  modules  or  “units 
of  action”)  (Simon,  1981).  Moreover,  he  argues  the  resulting  evolution  into  the 
complex  system  will  be  hierarchical.  Earlier,  in  The  Architecture  of  Complexity, 
Simon  (1962)  proposed  hierarchy  as  a  universal  principle  of  complex  structures. 
He  felt  that  complex  problems  could  be  solved  more  easily  when  decomposed 
into  sub-problems  (similar  to  how  project  managers  employ  Work  Breakdown 
Structures  via  the  Systems  Engineering  Process).  And,  sub-problem  solutions 
could  be  combined  into  a  solution  for  larger  problems,  etc. 

Commonly  seen  today  are  modular  industrial  products  that  are  sometimes 
designed  as  complete  architectures,  with  standardized  interfaces  that  invite 
others  to  introduce  complementary  products  for  insertion  (Agre,  2003).  The 
Modular  Open  Systems  Approach  (MOSA)  is  a  relatively  new  DoD  initiative  that 
encourages  the  use  of  widely  supported  commercial  interface  standards  and 
disciplined  interface  controls  to  develop  systems  architectures  using  modular 
design  concepts  (DoD  Open  Systems  Joint  Task  Force,  2004). 

As  in  biological  evolution,  improved  “fitness”  with  a  system’s  environment 
is  sought  in  the  adapting  or  evolving  of  systems.  But  others  have  noted  that 
Simon’s  metaphors  for  dynamic  complex  systems,  useful  as  they  are  for 
understanding  complexity,  fall  short  of  explaining  their  evolution.  While  the 
concept  of  modularity  suggests  approximately  independent  subsystems  may 
be  modified  or  adapted  as  such,  it  has  been  shown  that,  in  the  aggregate,  there 
is  yet  quantifiable  modular  interdependency  that  affects  evolvability  (Watson  & 
Pollack,  2005).  In  other  words,  how  changes  in  the  state  of  one  module  affect 
the  state  of  another  is  relative  and  measurable.  Thus,  Simon’s  writings  illustrate 
that  modularity  is  beneficial  for  production  but  not  necessarily  for  evolution. 

Examples  of  modular  interdependency  are  plentiful.  In  the  aircraft  or 
automotive  realm,  an  engine  upgrade  would  intuitively  seem  to  be  a  relatively 
independent  subsystem  change.  But,  systems  engineers  know  that  changes 
propagate  through  hardware  almost  as  much  as  software  in  the  long  run— just 
as  the  eventual  rise  in  building  temperature  from  the  thermostat  adjustment 
in  one  modular  room.  For  instance,  adding  increased  armor  protection  (and 
weight)  for  deployed  High  Mobility  Multi-purpose  Wheeled  Vehicles  has  resulted 
in  increased  wear-out  of  drive  train  and  suspension  components  and  impacts  to 
vehicle  range,  mobility,  mileage,  etc.  As  a  result,  “up  armor”  kits  have  become 
only  a  stopgap  measure  until  totally  redesigned  systems  can  be  produced. 
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Thus,  we  suggest  it  is  not  only  the  structural  modularity  and  standard 
interfaces  that  enable  system  evolution;  but,  it  is  also  the  relative  interdependency 
of  the  modules.  In  short,  PMs  need  to  be  mindful  of  the  degree  of  change  in 
subsequent  increments/spirals.  One  subsystem  is  likely  to  affect  another  in 
the  short-  or  long-run.  And,  that  can  make  product  evolution  problematic.  As 
Norman  Augustine  once  said,  “No  change  is  a  small  change”;  to  convey  that 
independent  subsystems,  even  redundant  ones,  aren’t  always  independent 
(Augustine,  1997). 


PRODUCTION  QUANTITY 

Many  might  correlate  the  applicability  of  evolutionary  development  with  long 
production  runs.  But  we  have  also  collaborated  with  officials  from  NASA  who 
have  said,  “No  two  identical  spacecraft  are  the  same,”  which  seems  to  contradict 
any  idea  that  like  configurations  are  a  necessity  among  small  production  lot  sizes 
(Roy,  2006).  Indeed,  naval  shipbuilders  voice  the  same  about  variation  among 
individual  ships,  or  within  flights,  of  the  same  class.  And  even  one-of-a-kind, 
nearly  immutable  projects  like  skyscrapers  and  bridges  can  be  later  remodeled 
and  refitted,  as  discussed  earlier.  Aside  from  truly  singular  efforts,  we  have  not 
yet  found  any  universal  evidence  of  an  evolutionary  approach  being  more  or 
less  suitable  according  to  quantity  of  systems  produced. 


Recommendations  for  Practice 

Project  managers  need  to  be  aware  of  the  inherent  risks  of  evolutionary 
development  and  take  necessary  precautions  to  balance  those  risks.  Many  tools 
and  control  measures  are  developed  and  available  to  assist  project  managers 
in  balancing  the  risks,  such  as  TRLs,  technical  performance  measurement, 
technical  reviews,  modeling  and  simulation,  real  options,  project  phasing,  risk 
management,  configuration  management,  earned  value  management,  and 
organizational  design. 

Incremental  development  projects  require  steps  to  alleviate  risks  that  may 
be  inherent  in  the  program  structure.  These  include  decisions  about  the  number 
and  concurrency  of  development  increments  and  their  scope  and  impact  on  the 
organization  staffing. 

Product  attributes  may  help  determine  the  suitability  of  evolutionary 
development.  PMs  should  consider  characteristics  such  as:  mutability,  time 
criticality,  man-rating,  modular  interdependency,  key  parameters  of  capability 
vs.  range  of  requirement  attainment  (i.e.,  binary  vs.  continuous),  and  the  relative 
amounts  of  modular  interdependency  in  the  system  architecture. 

Rigorous  configuration  management  accountability  must  be  assigned 
and  maintained  for  supportability,  reliability,  failure  mode  identification  and 
causality,  and  to  prevent  the  variety  generated  by  EA  from  reducing  total 
product  performance. 
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Conclusions 

Dr.  Barry  Boehm’s  recent  book  (2004)  on  software  development  advocates 
balancing  disciplined  (more  rigid)  and  agile  (more  flexible)  methods  to  capitalize 
on  the  benefits  of  both.  Discipline  is  needed  as  a  control  mechanism  to  avoid 
risk,  but  agility  is  needed  to  respond  quickly  to  customer  needs.  Saying,  “One 
size  fits  all  is  a  myth,”  he  advocates  a  balanced  approach  based  upon  risk. 
Consistent  with  our  findings,  he  also  advocates  more  disciplined,  risk-averse 
approaches  for  projects  that  are  mission/safety  critical,  larger  in  size,  and  have 
more  stable  requirements. 

It  could  be  summarized  that  evolutionary  development  was  at  its  inception, 
and  is  at  its  extension,  all  about  risk.  Paradoxically,  it  is  an  agile  method 
envisioned  to  reduce  risk  and  yet  can  potentially  add  its  own.  On  the  one  hand, 
a  spiral  or  incremental  approach  allays  risk  by  reducing  scope  to  render  only 
the  highest  priority  capabilities  with  the  exclusive  use  of  mature  technology; 
and  obtains  early  and  continuous  feedback  from  the  environment  for  follow- 
on  developments.  On  the  other  hand,  it  introduces  concurrency  during 
advanced  development  and  adds  variety  in  production,  with  all  their  attendant 
management  challenges. 

We  have  suggested  that  a  one-size-fits-all  methodology  for  DoD  system 
development  may  not  be  appropriate,  and  we  have  offered  for  consideration 
several  product  attributes  that  might  help  determine  the  applicability  of  agile 
approaches.  We  speculate  that  evolutionary  development  may  serve  better  than 
single-step  development  for  initial  capability  when  products  are  mutable,  time- 
critical,  non-maintenance-intensive,  and  have  continuous  (vs.  binary)  or  uncertain 
requirements,  short  cycle-times  (less  knock-on  effects),  sequentially  phased 
development  blocks,  and  modular  independence.  In  contrast,  evolutionary 
development  may  not  be  as  suitable  when  there  are  product  safety  or  man¬ 
rating  concerns  and  attributes  opposite  to  those  discussed  here.  In  particular, 
PMs  should  understand  the  nature  of  their  product  requirements  with  regard 
to  their  range  of  attainment  and  relative  to  key  parameters  of  capability  and 
vis-a-vis  the  readiness  level  of  their  enabling  technologies.  Some  key  features 
may  indeed  be  binary,  and  others  may  have  significant  ramifications  of  partial 
attainment— such  as  propagated  change  across  the  entire  product  componentry 
(as  in  weight  reduction)  vs.  a  more  independent  modular  modification. 

Variety  can  be  both  an  asset  (for  end-users)  and  a  liability  (for  manufacturers, 
owners,  and  supporters).  As  such,  to  compensate  for  product  variety  risk, 
we  posit  that  acquirers  must  “own”  the  design  and  emphasize  configuration 
management,  keeping  or  assigning  responsibility  for  that  function  and 
maintaining  accountability  for  it  (i.e.,  explanation  of  how  assigned  functions  are 
being  met). 

Our  title— “from  amorphous  to  defined”— alludes  not  only  to  product 
specification,  but  also  to  risk  realization  in  evolutionary  development.  PMs 
must  be  aware  it  has  inherent  challenges,  both  strategic  and  tactical;  they 
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must  balance  them  with  tools  that  we  have  mentioned.  In  this  article,  we  have 
both  highlighted  and  illustrated  them,  as  well  as  showing  that  incremental  and 
spiral  development  can  indeed  work  well— especially  for  technically  mature  and 
mutable  products  with  open  or  elegant  architecture. 

Finally,  stability  is  the  quest  in  all  things  programmatic:  for  funding, 
requirements,  design,  production  configuration,  etc.  But  in  an  unstable  world, 
and  with  the  future  filled  with  uncertainty,  the  only  constant  is  likely  to  be 
change,  and  tension  between  control  and  change  is  probably  unending.  PMs 
do  have  some  tools  for  coping,  and  being  forewarned  is  forearmed.  Successful 
use  of  these  tools  to  balance  control  and  risk  in  projects  with  a  high  rate  of 
change  and  concurrency  is  an  area  for  further  research,  to  improve  both  our 
understanding  and  use  of  evolutionary  acquisition. 
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WHAT  PERFORMANCE 
BASED  LOGISTICS  IS 
AND  WHAT  IT  IS  NOT- 
AND  WHAT  IT  CAN 
AND  CANNOT  DO 

(  Bill  Kobren 

Although  Performance  Based  Logistics  (PBL)  sustainment 
strategies  have  been  used  successfully  for  the  last  decade, 
misperceptions  persist.  This  article  discusses  what  PBL  is  and 
is  not;  and  what  it  can  and  cannot  do  for  the  military  services, 
program  managers,  and  ultimately  the  warfighter.  PBL  is  not 
about  contractors  on  the  battlefield  or  outsourcing  organic 
workload.  It  is  about  weapon  system  performance,  readiness, 
best  value  outcomes,  capability,  and  effective  and  efficient 
warfighter  support.  PBL  represents  a  fundamental  change 
in  how  DoD  supports  its  weapon  systems  and  ensures  those 
systems  are  reliable,  maintainable,  and  available  when  and 
where  the  warfighter  needs  them.  When  it  comes  to  deliv¬ 
ering  performance  outcomes:  PBL  works. 


Keywords:  Performance  Based  Logistics  (PBL), 
Performance  Based  Life  Cycle  Product  Support, 
Sustainment,  Product  Support 
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Performance  Based  Logistics  (PBL)  is  a  strategic  readiness  imperative.  As 
a  weapon  system  sustainment  strategy,  it  is  an  integral  mechanism  by  which 
the  Department  of  Defense  (DoD)  seeks  to  break  the  stranglehold  of  the  “death 
spiral,”  which  former  Under  Secretary  of  Defense  for  Acquisition,  Technology, 
and  Logistics,  Dr.  Jacques  Gansler,  warned  of  in  his  testimony  to  Congress 
earlier  this  decade. 

Our  equipment  is  aging.  We  cannot  replace  much  of  that  equipment  in 
the  near  future.  Consequently,  our  Operations  and  Maintenance  [O&M] 
costs  will  continue  to  escalate.  This  results  in  reduced  readiness— 
yet  at  increasing  costs.  And,  unless  we  reverse  the  trend  quickly  and 
deliberately,  we  face  what  I  have  described  as  a  “death  spiral”— a 
situation  where  reduced  readiness  requires  us  to  keep  removing  more 
and  more  dollars  from  equipment  modernization  and  putting  it  into 
daily  O&M,  thus  further  delaying  modernization,  causing  the  aging 
equipment  to  be  over-used,  further  reducing  readiness,  and  increasing 
O&M— a  vicious  circle.  (Gansler,  2000,  p.  68) 


Successful  PBL  support  strategies 

REPRESENT  A  WIN-WIN-WIN  FOR  THE 

WARFIGHTER,  ORGANIC  SUSTAINMENT  ORGANIZATIONS, 

AND  INDUSTRY  PARTNERS. 


By  leveraging  long-term  performance  based  agreements  and  incentivizing 
desired  outcomes  using  a  well-crafted  set  of  metrics,  PBL  can  deliver  substantial 
performance  improvements  for  both  new  and  legacy  weapon  systems  over 
traditional  “spares  and  repairs”  sustainment  models.  Moreover,  when  these 
strategies  are  properly  implemented,  the  resultant  outcomes  can  often  be 
achieved  at  a  lower  cost  than  otherwise  attained  through  more  traditional 
sustainment  approaches. 

Despite  the  fact  PBL  support  and  sustainment  strategies  have  been 
successfully  used  by  the  department  for  almost  ten  years,  however, 
misperceptions  persist  within  the  DoD  acquisition,  logistics,  and  sustainment 
communities  as  to  exactly  what  this  thing  called  PBL  is  all  about.  This  article  will 
qualitatively  examine  a  range  of  documentation  on  the  subject  in  an  attempt 
to  clarify  what  PBL  is  and  is  not,  and  perhaps  just  as  importantly,  what  it  can 
and  cannot  do  for  the  department,  the  military  services,  the  weapon  system 
program  manager,  and  ultimately  the  warfighter;  additionally,  it  will  examine 
some  of  the  key  strategic  implications  for  the  DoD  logistics  and  sustainment 
community  charged  with  supporting  aging  weapon  systems  in  an  increasingly 
austere  budgetary  environment. 
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What  PBL  Is 

So  what  exactly  is  this  thing  called  PBL?  Simply  put,  PBL  is: 


FIRST  AND  FOREMOST,  ABOUT  SUPPORTING  THE  WARFIGHTER 

PBL  is  about  performance.  It  is  about  readiness.  It  is  about  enabling  mission 
accomplishment  and  ensuring  the  warfighter  has  weapon  systems  that  are 
available,  reliable,  and  supportable  when  and  where  required. 


A  WEAPON  SYSTEM  SUPPORT  STRATEGY 

As  stated  in  the  Defense  Acquisition  Guidebook,  "Performance  Based 
Logistics  (PBL)  is  DoD’s  preferred  approach  for  product  support  implementation 
{DAG,  2006,  p.  DAG-196).  Succinctly,  PBL  is  defined  as  “the  purchase  of  support" 
as  an  integrated,  affordable,  performance  package  designed  to  optimize  system 
readiness  and  meet  performance  goals  for  a  weapon  system  through  long-term 
support  arrangements  with  clear  lines  of  authority  and  responsibility”  (DAG, 
2006,  p.  DAG-196). 


DoD  POLICY 

“PMs  [Program  Managers]  shall  develop  and  implement  performance  based 
logistics  strategies  that  optimize  total  system  availability  while  minimizing  cost 
and  logistics  footprint”  (Department  of  Defense  [DoD],  2008,  p.  7). 


FOCUSED  ON  PERFORMANCE  OUTCOMES  RATHER  THAN  DISCRETE 
TRANSACTIONS 

Instead  of  relying  on  a  traditional  “spares  and  repairs”  sustainment  model,  “the 
essence  of  Performance  Based  Logistics  is  buying  performance  outcomes”  {DAG, 
2006,  p.  DAG-197).  It  is  procurement  of  a  capability  to  support  the  warfighter  vs. 
“the  individual  parts  or  repair  actions”  {DAG,  2006,  p.  DAG-197). 


A  FACILITATOR  OF  PUBLIC-PRIVATE  PARTNERING  (PPP)  INITIATIVES 

PBL  support  strategies  “shall  include  the  best  use  of  public-  and  private- 
sector  capabilities  through  government/industry  partnering  initiatives,  in 
accordance  with  statutory  requirements”  (DoD,  2003,  p.  7).  Successful 
PBL  support  strategies  represent  a  win-win-win  for  the  warfighter,  organic 
sustainment  organizations,  and  industry  partners. 
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AN  IMPORTANT  TOOL  FOR  MINIMIZING  LIFE  CYCLE  COSTS 

If  properly  implemented,  with  carefully  constructed  and  clearly  understood 
metrics,  incentive  structure,  financial  construct,  and  (if  appropriate)  contracting 
strategy,  “Performance  Based  Logistics  can  help  program  managers 
optimize  performance  and  cost  objectives  [including]  through  the  strategic 
implementation  of  varying  degrees  of  Government-Industry  partnerships” 
(DAG,  2006,  p.  DAG-196). 


TAILORABLE  TO  THE  UNIQUE  NEEDS  OF  EACH  INDIVIDUAL  PROGRAM 

“Although  the  fundamental  concept  of  buying  performance  outcomes  is 
common  to  each  PBL  arrangement,  the  PBL  strategy  for  any  specific  program 
or  commodity  must  be  tailored  to  the  operational  and  support  requirements  of 
the  end  item”  (Defense  Acquisition  University  [DAU],  2005a,  p.  2-4).  “There  is 
no  one-size-fits-all  approach  to  PBL.  Similarly,  there  is  no  template  regarding 
sources  of  support  in  PBL  strategies.  Almost  all  of  DoD’s  system  support 
comprises  a  combination  of  public  (organic)  and  private  (commercial)  support 
sources”  (DAU,  2005a,  p.  2-4). 


FOCUSED  ON  BEST  VALUE,  INCLUDING,  BUT  NOT  NECESSARILY  LIMITED  TO 
LOWEST  COST 

“Finding  the  right  mix  of  support  sources  is  based  on  best  value 
determinations  of  inherent  capabilities  and  compliance  with  statutes  and  policy. 
This  process  will  determine  the  optimum  PBL  support  strategy  within  the 
product  support  spectrum,  which  can  range  from  primarily  organic  support  to 
a  total  system  support  package  provided  by  a  commercial  Original  Equipment 
Manufacturer  (OEM)”  (DAU,  2005a,  p.  2-4).  The  exact  definition  of  what 
actually  constitutes  a  best  value  support  solution  often  varies  from  program  to 
program,  but  along  with  a  cost  component,  frequently  will  also  include  some 
combination  of  performance,  capability,  skills,  infrastructure,  flexibility,  quality, 
reliability,  integration,  and  maintainability,  among  other  components.  Successful 
achievement  of  these  best  value  outcomes  is  largely  determined  by  the  metrics 
and  incentives  identified  in  the  PBL  product  support  strategy. 
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What  PBL  Is  Not 

Conversely,  there  are  also  some  things  PBL  cannot  claim  to  be.  For  example, 
PBL  is  not: 


A  NEW  CONCEPT  OR  A  “FLAVOR  OF  THE  MONTH"  INITIATIVE 

The  roots  of  DoD  PBL  policy  date  back  more  than  a  decade,  articulated 
in  Section  912(c)  of  the  National  Defense  Authorization  Act  (NDAA)  for  FY 
1998,  and  the  April  1998  Secretary  of  Defense  Report  to  Congress:  Actions  to 
Accelerate  the  Movement  to  the  New  Workforce  Vision  in  response  to  Section 
912(c)  of  the  NDAA  for  FY  1998.  This  report  formed  the  basis  for  the  July  1999 
Product  Support  for  the  2ist  Century:  Report  of  the  Department  of  Defense 
(DoD)  Product  Support  Reengineering  Implementation  Team  Section  9i2c\  the 
September  2000  Product  Support  for  the  21st  Century:  A  Year  Later,  and  the 
November  2001  Product  Support  for  the  21st  Century:  A  Program  Manager’s 
Guide  to  Buying  Performance.  PBL  guidance  was  codified  in  the  May  2003 
DoD  Directive  5000.01,  The  Defense  Acquisition  System,  and  DoD  Instruction 
5000.02,  Operation  of  the  Defense  Acquisition  System]  and  supported  by  detailed 
implementation  guidance  contained  in  Chapter  5  of  the  Defense  Acquisition 
Guidebook  (DAG)  in  2006,  issuance  of  Performance  Based  Logistics:  A  Program 
Manager’s  Product  Support  Guide  in  March  2005,  and  numerous  related 
Office  of  the  Secretary  of  Defense  (OSD)  and  Service  policies,  instructions, 
regulations,  and  guidebooks.  At  OSD  direction,  DAU  also  created  a  series  of 
PBL-related  learning  assets,  including  Continuous  Learning  Module  (CLM  Oil) 
Performance  Based  Logistics  (PBL);  LOG  235A  (now  LOG  235)  Web-based  PBL 
training;  LOG  235B  (now  LOG  236)  case-based  classroom  PBL  training;  and 
establishment  of  the  Web-based  PBL  toolkit  (https://acc.dau.mil/pbl)  in  2005. 

In  October  2005,  “consistent  with  the  Defense  Business  Board 
recommendation  to  leverage  DAU  to  accelerate  PBL  implementation  and  to 
establish  a  DoD  PBL  Center  of  Excellence”  (DAU,  2005b,  p.  1),  the  Assistant 
Deputy  Under  Secretary  of  Defense,  Logistics  Plans  and  Programs  designated 
DAU  as  a  PBL  “Center  of  Excellence”  (DAU,  2005b,  p.  1),  to  expand  PBL  learning 
assets,  performance  support,  workshops,  rapid  deployment  training,  and 
“serve  as  a  nexus  for  information  cross-flow,  liaison,  and  interface  between 
and  among  the  DoD  components,  the  Defense  Industry,  and  other  Academic 
institutions  on  PBL  applications  and  thought  leadership”  (DAU,  2005b,  p.  1).  In 
fact,  the  Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology 
and  Logistics  (USD[AT&L])  was  so  serious  about  PBL  success,  that  the  under 
secretary  established  an  annual  DoD-level  awards  program  in  2005  to  recognize 
outstanding  system,  sub-system,  and  component-level  PBL  strategies  across 
the  DoD.  This  compendium  of  policies,  guidance,  initiatives,  training  structures, 
and  program  recognition  attests  to  the  fact  that  PBL  is  clearly  not  a  passing  fad. 
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OUTSOURCING  OR  CONTRACTOR  LOGISTICS  SUPPORT  (CLS) 

To  repeat:  PBL  is  not  synonymous  with  outsourcing  or  contractor  logistics 
support.  This  is  clearly  articulated  in  the  new  December  2008  DoD  Instruction 
5000.02 :  “PBL  offers  the  best  strategic  approach  for  delivering  required  life 
cycle  readiness,  reliability,  and  ownership  costs.  Sources  of  support  may  be 
organic,  commercial,  or  a  combination  [emphasis  added],  with  the  primary 
focus  on  optimizing  customer  support,  weapon  system  availability,  and  reduced 
ownership  costs”  (p.  29).  While  a  majority  of  successful  PBL  Product  Support 
Integrators  (PSI)  are  in  fact  industry  partners  (and  in  many  cases,  the  OEM), 
contrary  to  popular  misconception,  there  is  no  mandate  in  DoD  policy  to  use 
a  commercial  sector  PSI,  or  even  use  an  industry  product  support  provider 
(PSP).  “Part  of  the  reason  for  this  [mis]perception  is  that  contractors  have  been 
effective  and  integral  to  most  of  the  PBL  strategies  employed  to  date.  PBL  has 
not  significantly  changed  DoD’s  reliance  on  contractors;  it  has  only  changed  the 
nature  of  how  we  use  their  services”  (Fowler,  2009,  p.  10). 

In  reality,  PBL  optimizes  the  best  public-  and  private-sector  competencies 
“based  upon  a  best-value  determination,  evidenced  through  a  business  case 
analysis  (BCA),  of  the  provider’s  product  support  capability  to  meet  set 
performance  objectives”  {DAG,  2006,  p.  DAG-197).  This,  as  expressed  in  the 
following  excerpt  from  the  Defense  Acquisition  Guidebook,  is  absolutely  critical 
to  understand: 

This  major  shift  from  the  traditional  approach  to  product  support 
emphasizes  how  program  manager  teams  buy  support,  not  who  they 
buy  from  [emphasis  added].  Instead  of  buying  set  levels  or  varying 
quantities  of  spares,  repairs,  tools,  and  data,  the  focus  is  on  buying  a 
predetermined  level  of  availability  to  meet  the  warfighter’s  objectives. 
{DAG,  2006,  p.  DAG-197) 

While  the  authors  of  the  Defense  Acquisition  Guidebook  Chapter  5  could 
arguably  have  avoided  confusion  by  choosing  a  different  word  such  as  procure 
or  obtain,  rather  than  ‘buy’,  it  is  a  fact  that  “effective  PBL  requires  balanced 
contribution  by  both  public-  and  private-sector  providers”  (Fowler,  2009,  p.  10). 


A  PANACEA 

PBL  will  not  overcome  a  lack  of  sustainment  planning,  make  up  for  an  absence 
of  effective  program  systems  engineering,  succeed  with  inadequate  funding, 
mitigate  the  effects  of  poor  leadership,  or  deliver  instantaneous  results.  “PBL 
can  often  improve  reliability,  but  there  are  limitations,  particularly  on  legacy 
systems.  Long-standing,  systemic  reliability  problems  in  fielded  systems  are 
unlikely  to  be  corrected  without  appropriate  commitment  of  necessary  funding” 
(L.  Garvey,  personal  communication,  November  27,  2008).  The  key  is  to  establish 
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solid,  well  crafted,  integrated  metrics  and  incentives  emphasizing  the  desired 
performance  outcomes,  most  notably  (but  certainly  not  limited  to)  readiness, 
reliability,  availability,  maintainability,  cost,  and  obsolescence/Diminishing 
Manufacturing  Sources  and  Material  Shortages  (DMSMS)  mitigation.  To  use  a 
baseball  analogy,  DoD  program  managers  and  life  cycle  logisticians  alike  must 
recognize  that  ignoring  early  logistics  design  influence  opportunities  cannot  be 
rescued  by  a  PBL  “diving  basket  catch”  at  the  eleventh  hour. 


APPROPRIATE  FOR  EVERY  SYSTEM 

In  some  instances,  particularly  for  legacy  systems  approaching  retirement, 
PBL  may  in  fact  not  be  the  most  appropriate  support  solution.  In  other  instances, 
the  organic  sector  may  be  unable  to  effectively  or  efficiently  support  a  system 
or  the  commercial  sector  may  be  unwilling  to  invest  in  such  a  strategy,  judging 
the  risks  to  be  too  great  or  the  returns  to  be  too  inadequate.  Only  through  a 
well-crafted,  program-specific,  and  periodically  updated  business  case  analysis 
process  can  the  program  manager  confidently  make  this  determination. 


STATIC 

PBL  policies,  best  practices,  implementation  strategies,  and  training 
continue  to  evolve  as  DoD  and  industry  better  understand  the  successes, 
challenges,  obstacles,  and  issues  related  to  PBL  implementation  and  execution. 
New  policy  guidance  was  issued  in  a  July  2008  policy  memorandum  by  the 
USD(AT&L),  for  example,  which  states: 

For  several  years,  acquisition  and  sustainment  management  [has] 
been  appropriately  focused  on  performance-based  strategies.  DoD 
Directive  5000.1  currently  recognizes  performance  based  logistics 
(PBL)  as  a  key  policy  principle.  I  direct  the  Secretaries  of  the  Military 
Departments  to  continue  this  emphasis  with  a  more  precise  orientation 
on  life  cycle  product  support  [emphasis  added],  PBL  offers  the  best 
strategic  approach  for  delivering  readiness,  reliability,  and  reduced 
ownership  costs.  All  of  the  policies  and  directions  discussed  in  this 
memorandum  are  enabled  by  effective  PBL  implementation.  I  want 
to  emphasize  that  PBL  is  not  a  contracting  strategy— it  is  indeed  a 
strategy  applicable  to  both  private  sector  and  DoD  organic  providers. 
To  facilitate  effective  PBL  implementation,  I  direct  the  DUSD  (L&MR) 
[Deputy  Under  Secretary  of  Defense,  Logistics  and  Materiel  Readiness] 
to  reflect  appropriate  procedural  strengthening  in  the  Defense 
Acquisition  Guidebook.  I  further  direct  that  all  MDAPs  [Major  Defense 
Acquisition  Programs]  reflect  PBL  implementation  approaches  in  Life 
Cycle  Sustainment  planning.  (Young,  2008,  p.  3) 
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Newly  issued  DoD  Instruction  5000.02  language  reiterates  the  shift  in  focus 
from  (performance  based)  logistics  to  (performance  based)  life  cycle  product 
support,  stating: 

The  PM  shall  work  with  the  user  to  document  performance  and 
sustainment  requirements  in  performance  agreements  specifying 
objective  outcomes,  measures,  resource  commitments,  and  stakeholder 
responsibilities.  The  PM  shall  employ  effective  Performance  Based  Life 
Cycle  Product  Support  (PBL)  planning,  development,  implementation, 
and  management.  Performance  Based  Life  Cycle  Product  Support 
represents  the  latest  evolution  of  Performance  Based  Logistics.  "Both 
can  be  referred  to  as  “PBL.”  (DoD,  2008,  p.  29) 

Further  emphasizing  how  PBL  policy  and  practices  are  not  static,  DoD 
policy  makers  established  a  Product  Support  Assessment  study  team  in 
September  2008  (DUSD[L&MR],  2008),  assembling  participants  from  across 
the  department  to  examine  what  a  next  generation  PBL  arrangement  might 
look  like;  in  particular,  should  the  PBL  business  model  be  refined?  In  light  of 
current  economic  and  DoD  budget  pressures,  life  cycle  cost  reductions  will  very 
likely  continue  to  be  of  paramount  interest  in  the  next  evolution  of  the  PBL 
business  model. 


(NECESSARILY)  A  TWO-LEVEL  (ORGANIZATIONAL-TO-DEPOT)  MAINTENANCE 
STRATEGY 

The  operative  word  here  is  “necessarily.”  While  many  successful  PBL 
arrangements  leverage,  facilitate,  or  encourage  a  two-level  maintenance 
strategy,  a  two-level  maintenance  strategy  is  not  a  requirement  for,  a  definition 
of,  or  synonymous  with  a  PBL  support  strategy.  In  fact,  “many  PBLs  effectively 
(sustain)  and  enhance  systems  supported  with  three  levels  of  maintenance” 
(L.  Garvey,  personal  communication,  November  27,  2008).  This  is  particularly 
true  for  PBL  strategies  implemented  for  previously  fielded  legacy  systems, 
which  were  very  often  developed  years  or  even  decades  ago  with  a  three- 
level  maintenance  strategy  that  included  an  intermediate  level  back-shop 
maintenance  requirement. 
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What  PBL  Can  Do 

So  what  exactly  can  PBL  do  for  a  weapon  system  program  manager  and  his 
or  her  staff?  PBL  can: 


DELIVER  HIGHLY  EFFECTIVE  SYSTEM,  SUB-SYSTEM,  OR  COMPONENT 
SUSTAINMENT 

“Performance  Based  Logistics,  a  strategy  for  making  sure  warfighters  have 
the  equipment  they  need  when  they  need  it,  (quite  simply)  works.  Government, 
industry  and  academic  studies  show  PBL  contracts  regularly  improve  availability 
20-40%  and  (reduce)  costs  by  15-20%”  (Miller,  2008,  p.  78).  PBL  delivers 
results.  VADM  Walter  Massenberg,  Naval  Air  Systems  Command  (NAVAIR) 
Commander  clearly  reiterated  this  point  in  a  February  2007  memo  entitled  “PBL 
Guidance  and  Best  Practices"  where  he  categorically  stated  that  “the  success  of 
Performance  Based  Logistics  (PBL)  has  allowed  the  Naval  Aviation  Enterprise 
to  improve  support  to  the  warfighter  and  achieve  weapon  system  readiness  at 
lower  life  cycle  costs”  (Massenburg,  2007,  p.  1). 


INCENTIVIZE  DESIRED  BEHAVIOR 

Both  NAVAIR  and  Naval  Supply  Systems  Command  (NAVSUP)  have 
experienced  substantial  success  in  implementing  PBL  arrangements.  Their 
philosophy  is  simple:  The  Navy  buys  [a]  comprehensive  performance  package... 
not  individual  parts.  This  approach  totally  reverses  [the]  vendor  incentive. 
Fixed  price  “pay  for  performance”  contracts  motivate  [the]  vendor  to  reduce 
failures/consumption,  [while]  a  long-term  commitment  enables  [the]  vendor  to 
balance  risk  versus  investments.  [This  in  turn]  improves  parts  support  (Material 
Availability  increases  and  Logistics  Response  Time  [LRT]  decreases,  resulting  in 
improved  readiness);  optimizes  depot  efficiency  by  reducing  Repair  Turn  Around 
Times  (RTAT),  Awaiting  Parts  (AWP),  and  Work  in  Process  (WIP);  [incentivizes] 
investments  in  reliability,  [resulting  in]  Mean  Time  Between  Failures  (MTBF) 
[improvement];  and  shortstops  failures  [in  turn]  reducing  off-station  demand 
(Garvey,  2004). 


HELP  THE  PM  STREAMLINE  SUPPORT  STRATEGY  DEVELOPMENT 

Randy  Fowler  (2009)  described  the  properties  of  PBL  in  their  most 
fundamental  sense: 

PBL,  with  its  outcome-focused  principles,  metrics,  and  incentives,  serves 
as  a  simplifying  strategy  for  the  PM.  PBL  offers  a  one-stop  approach  for 
the  PM  to  perform  effectively  as  the  life  cycle  manager.  PBL  is  the  best 
enabler  of  the  total  life  cycle  systems  management  concept;  it  provides 
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a  means  for  the  resource-constrained  program  management  office  to 
develop,  implement,  and  manage  the  sustainment  of  a  system  over  its 
life  cycle,  (p.  12) 


BE  APPLIED  FLEXIBLY  DEPENDING  ON  A  PROGRAM'S  UNIQUE  NEEDS 

Application  of  “Performance  Based  Logistics  strategies  may  be  at  the 
system,  subsystem,  or  major  assembly  level  depending  on  program  unique 
circumstances  and  appropriate  product  support  strategy  analysis”  (DAG,  2006, 
p.  DAG-177). 


SERVE  AS  A  CRITICAL  TOOL  IN  THE  TOOLKIT  FOR  PROACTIVELY  MITIGATING 
DMSMS  AND  OBSOLESCENCE  ISSUES 

PBL  offers  an  effective  way  to  deal  with  obsolescence  throughout  the 
life  of  a  product.  Unlike  traditional  approaches  to  modernizing  legacy 
systems,  PBL  holistically  manages  the  product  support  of  weapon 
systems,  assemblies,  subassemblies,  and  components.  As  the  point 
of  responsibility  for  meeting  performance  requirements,  as  outlined 
in  the  Performance  Based  Agreement,  shifts  to  the  Product  Support 
Integrator  (PSI)  under  the  Program  Manager,  PBL  provides  a  powerful 
tool  for  mitigating  obsolescence  and  making  continuous  modernization 
(CM)  a  reality  for  current  weapon  systems,  assemblies,  subassemblies, 
and  components  (where  a  PBL  application  is  feasible).  PBL  clearly 
fulfills  the  need  for  CM  and  obsolescence  mitigation.  (DoD,  2006,  p.  2-1) 


SERVE  AS  AN  IMPORTANT  ENABLER  OF  AN  EFFECTIVE,  END-TO-END 
SUPPLY  CHAIN 

“Performance  Based  Logistics  enables  the  program  manager  to  exploit 
supply  chain  processes  and  systems  to  provide  flexible  and  timely  materiel 
support  response  during  crises  and  joint  operations”  (DAG,  2006,  p.  DAG-184). 

A  Supply  Chain  Management  (SCM)  strategy  is  critical  to  the  success 
of  any  PBL  effort.  Materiel  support  is  a  critical  link  in  weapons  systems 
supportability.... Supply  chain  management  includes  the  distribution, 
asset  visibility,  and  obsolescence  mitigation  of  the  spare  parts.  From 
a  warfighter’s  perspective,  transportation  and  asset  visibility  have  a 
substantial  impact  on  high-level  metrics  and  should  be  emphasized  in 
the  PBL  strategy.  (DAU,  2005a,  pp.  3-7,  3-8) 
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POWERFULLY  INCENTIVIZE  THE  WEAPON  SYSTEM  PSI  TO  INVEST  IN  MAJOR 
RELIABILITY,  AVAILABILITY,  AND  MAINTAINABILITY  INITIATIVES 

Substantial  improvements  in  system  and  subsystem  reliability,  time-on- 
wing,  and  operational  availability  have  been  seen  on  a  variety  of  programs 
which  have  implemented  PBL  support  strategies. 

PBL  inherently  self-motivates  service  providers  to  do  “good  things,” 
such  as  improve  component  and  system  reliability,  since  it  provides 
the  foundation  for  increased  profit.  However,  this  motivation  must  be 
balanced  against  the  ability  of  the  service  provider  to  invest  in  the 
needed  infrastructure  and  processes  required  to  achieve  reliability 
improvements.  (DAU,  2005a,  p.  3-10) 


What  PBL  Cannot  Do 

On  the  other  hand,  however,  PBL  cannot  be  all  things  to  all  people  (or  all 
programs).  It  cannot,  for  example: 


OVERCOME  POOR  SUSTAINMENT  PLANNING,  LACK  OF  ADEQUATE  TRAINING, 
OR  A  MISREPRESENTATION  OF  WHAT  PBL  IS 

Kate  Vitasek  and  Steve  Geary  (2008)  examined  the  reasons  why  some 
managers  fail  to  implement  PBL  successfully  and  came  to  the  following 
conclusion: 

Most  thought  leaders  agree  that  the  PBL  business  model  works,  but  not 
all  programs  have  lived  up  to  the  success  they  hoped  to  achieve.  Why  is 
this?  Many  point  to  poor  application  of  the  PBL  concepts. 

A  report  of  the  Acquisition  Advisory  Panel  sums  it  up  best:  ‘When 
individuals  without  the  proper  training  and  experience  attempt 
to  implement  a  performance-based  contract,  the  results  are 
understandably  and  expectedly  poor.. .there  is  trouble  consistently 
implementing  it  by  an  inconsistently  trained  workforce,  (p.  64) 


RELIEVE  THE  PROGRAM  MANAGER  OF  HIS  OR  HER  RESPONSIBILITIES  AS  LIFE 
CYCLE  MANAGER  FOR  THE  SYSTEM 

“The  Program  Manager  [is]  charged  with  responsibility  for  supporting  the 
entire  system. ...The  scope  of  support  accountability  for  a  PM  never  varies— they 
are  responsible  for  supporting  the  entire  system”  (Cothran,  2007,  p.  3). 
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Conclusions 

In  conclusion,  it  is  important  to  understand  what  PBL  is  and  is  not. 
Additionally,  while  there  are  many  things  PBL  can  and  cannot  do,  it  remains 
firmly  entrenched  as  a  major  initiative  and  part  of  the  acquisition  process. 
Randy  Fowler  (2009),  in  an  article  published  in  the  Defense  Acquisition 
University’s  Defense  AT&L  periodical,  made  the  case  for  PBL’s  contribution  to 
the  acquisition  process: 

The  evidence  is  clear:  PBL  works.  PBL  delivers  dramatic  improvements 
in  performance  with  lower  operating  costs  across  the  total  life  cycle. 
PBL  does  more  for  the  warfighter  with  less  from  the  taxpayer.  Instead 
of  paying  for  transactional  activities,  the  government  and  industry 
partners  deliver  improved  performance  at  lower  costs,  (p.  13) 

At  the  end  of  the  day,  PBL  is  not  about  contractors  on  the  battlefield, 
outsourcing,  degrading  organic  workforce  expertise,  or  taking  workload  away 
from  organic  maintenance  depots.  It  is  about  weapon  system  performance.  It 
is  about  readiness,  best  value  outcomes,  capability,  and  providing  effective  and 
efficient  support  for  the  warfighter.  PBL  represents  a  fundamental  change  in 
how  DoD  supports  its  weapon  systems  and  ensures  those  systems  are  reliable, 
maintainable,  and  perhaps  most  importantly,  available  when  and  where  the 
warfighter  needs  them,  in  the  most  cost-effective  manner  possible.  Ultimately, 
this  is  what  PBL  can— and  must  continue  to  do— for  our  warfighters  and  our  nation. 
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JOINT  ACQUISITION 
COMMAND  DOCTRINE 
-A  SUCCESS  STORY 


\'  Al  Borzoo,  Constance  S.  Short,  Ken  Brockway, 
and  Col  Stan  L.  VanderWerf,  USAF 

In  the  past,  war  planners  typically  treated  acquisition  as 
an  afterthought.  Operations  Iraqi  Freedom  and  Enduring 
Freedom  Joint  Task  Force  commanders  were  not  initially 
afforded  the  full  value  of  acquisition  capabilities  to  buy  local 
resources  and  manage  the  exploding  number  of  contractors 
in  the  Area  of  Responsibility  (AOR).  Recognizing  this  short¬ 
fall,  DoD  created  the  Joint  Contracting  Command  (JCC).  The 
JCC  provided  substantial  contracting  capacity  and  coordina¬ 
tion-critical  attributes  to  effective,  efficient  AOR  operations. 

This  research  and  resultant  report,  originally  prepared  in 
2006  for  the  Industrial  College  of  the  Armed  Forces,  played 
a  substantial  role  in  shaping  joint  thinking,  culminating  in 
creating  Joint  Publication  (JP)  4-10,  Operational  Contract 
Support.  JP  4-10  establishes  long-needed  joint  contracting 
doctrine  for  Combatant  Command  AOR  operations. 


Keywords:  War  Planning,  Joint  Contracting, 
Acquisition  Capabilities,  Joint  Contracting 
Command,  Joint  Publication  4-10 
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Joint  Acquisition  Command  Doctrine 
—Yes  It's  Needed! 

"The  Department’s  Total  Force— its  active  and  reserve  military  components, 

its  civil  servants,  and  its  contractors— constitutes  its  warfighting  capability 

and  capacity." 

Quadrennial  Defense  Review  Report,  2006,  p.  74 

In  today’s  fast-moving  military  environment,  contractors  have  become 
critical  to  Joint  Task  Force  (JTF)  operations.  As  a  result,  contract  management 
increased  in  complexity  and  scope  requiring  robust  Department  of  Defense 
(DoD)  management  to  maximize  the  effectiveness  and  efficiency  of  local 
material  procurement  and  the  contractor’s  support  to  the  mission. 

As  operations  in  Iraq  were  to  prove,  insufficient  contractor  support  planning 
placed  great  strains  on  the  United  States’  ability  to  manage  its  contractors. 
While  contracting  efforts  were  ultimately  successful,  insufficient  contract 
management  capacity  and  coordination  led  to  substantial  efficiency  losses  and 
a  reduction  in  effectiveness. 

In  recognizing  these  deficiencies,  a  Joint  Contracting  Command  (JCC) 
with  Joint  Service  participation  and  theater-wide  responsibility  was  created 
for  managing  contracting  efforts.  However,  doctrine  did  not  exist  for  such 
an  organization.  Identifying  this  shortfall  and  in  an  attempt  to  address  these 
deficiencies,  we  conducted  research  to  examine  the  feasibility  of  establishing 
joint  contracting  doctrine  for  Combatant  Command  Area  of  Responsibility 
(COCOM  AOR)  operations.  Along  with  this  research,  we  submitted  a  report  to 
the  Industrial  College  of  the  Armed  Forces  in  2006  that  proposed  codification 
of  joint  contracting  doctrine  to  permanently  capture  JTF  lessons  learned  and 
ensure  adequate  deliberate  contract  planning  for  Operational  Plans  (OPLANs) 
and  Contingency  Plans  (CONPLANs).  The  report  proposed  a  full  range  of 
acquisition  processes  to  accommodate  the  increasing  workload  of  contractors 
in  the  AOR. 

We  also  submitted  the  report  to  the  Joint  Staff  in  2006,  where  the  J-4 
offices  took  great  interest  in  our  recommendations  for  joint  doctrine.  On 
October  17,  2008,  Joint  Publication  4-10,  Operational  Contract  Support,  was 
officially  published  (JCS,  2008).  JP  4-10  addresses  every  concern  presented  in 
our  report,  places  in  doctrine  almost  every  recommendation  from  our  research, 
and  reaches  even  further  into  a  war  planning  option  only  recently  applied  on 
the  battlefield  during  Operations  Iraqi  Freedom  and  Enduring  Freedom:  making 
maximum  use  of  contractors  on  the  battlefield. 

The  success  of  our  efforts  validated  our  research  and  reinfo  reed  a  professional 
ethos  to  which  we  believe  acquisition  practitioners  and  professionals  would  do 
well  to  adhere:  If  you  suspect  that  an  activity,  task,  or  policy  is  not  correct,  then 
you  should  take  action.  Investigate  it,  write  about  it,  and  have  the  fortitude  to 
make  changes. 
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PAST  JTF  CONTRACT  PLANNING 

In  JTFs  prior  to  and  during  Iraq,  the  Services  brought  their  own  contracting 
capabilities.  Integration  of  the  Service-let  contracts,  if  it  took  place,  was 
not  by  design.  With  increasing  acquisition  requirements  placed  on  DoD  for 
nontraditional  reconstruction  and  stabilization  operations  as  seen  in  Iraq,  no 
one  Service  or  agency  initially  had  responsibility.  Contracting  officers  often 
failed  to  use  cost-effective  local  capabilities,  opting  instead  for  the  Logistics 
Civil  Augmentation  Program  (LOGCAP),  even  when  local  vendors  served 
to  reduce  DoD  logistics  requirements  (S.  M.  Seay,  personal  communication, 
January  24,  2006).  At  other  times,  the  Services  unknowingly  competed  against 
each  other  for  local  resources.  These  and  other  issues  illustrated  the  need  for 
an  overarching  acquisition  strategy— one  that  would  meet  effectiveness  and 
efficiency  goals  as  well  as  policy  requirements  for  the  JTF.  The  Government 
Accountability  Office  (GAO)  recognized  this  as  a  systemic  problem  as  far  back 
as  the  late  1980s.  The  GAO  recommended  DoD  include  contract  management 
in  all  operations  planning  (GAO,  2004b,  p.  5). 


DOCTRINE  PREVENTS  REPEATING  MISTAKES 

The  best  reason  for  doctrine  is  to  codify  the  lessons  learned  from  past 
mistakes.  The  data  collected  for  this  study  showed  the  DoD  suffered  from  a  lack 
of  operational  contracting  doctrine  during  contingency  and  post-contingency 
operations  at  least  as  far  back  as  July  1992,  when  the  U.S.  involvement  in  Bosnia 
began  in  the  Balkans  as  part  of  humanitarian  relief  efforts  (GAO,  2000). 

Our  research  found  the  DoD  learned  and  relearned  these  lessons  at  each 
major  contingency  despite  the  fact  that  lessons  learned  generated  after 
each  conflict  demonstrated  the  need  for  doctrine.  The  case  for  doctrine 
was  compelling  (D.  A.  Scott,  personal  communication,  January  4,  2006;  C. 
M.  Bolton,  personal  communication,  January  19,  2009;  S.  M.  Seay,  personal 
communication,  January  24,  2006;  L.  H.  Thompson,  personal  communication, 
February  9,  2006;  &  M.  J.  Brown,  personal  communication,  March  1,  2006).  Joint 
Pub  4-10  also  supports  this  assertion  in  its  Introduction,  which  includes  quotes 
from  the  2006  Quadrennial  Defense  Review  (QDR)  and  a  U.S.  Marine  Corps 
statement  of  contractor  support  in  World  War  II. 


CORE  RESEARCH  RECOMMENDATIONS  AND  HOW  THEY  COMPARE  TO  JP4-10 
Our  research,  conducted  at  the  Industrial  College  of  the  Armed  Forces 
(ICAF)  in  2006  for  the  Acquisition  School,  culminated  in  an  82-page  report 
titled  Joint  Acquisition  Command  Doctrine.  Using  historical  research,  interviews, 
and  other  sources,  it  offered  recommendations  to  codify  joint  contracting.  At 
that  time,  our  J-4  point  of  contact  was  tasked  with  initiating  the  development 
of  Joint  Doctrine  for  JTF  integrated  contracting.  During  our  research,  we  held 
several  meetings  with  action  officer  representatives  from  J-4,  Joint  Chiefs  of 
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Staff  (JCS).  By  mid-2007,  a  full  draft  joint  contracting  doctrine  publication 
was  under  formal  review.  Of  note,  our  report  generated  26  recommendations 
for  joint  contracting  doctrine— 24  of  which  are  found  in  JP  4-10.  The  following 
discussion  pinpoints  selected  descriptions  of  our  study’s  recommendations  and 
how  JP  4-10  addressed  those  concepts. 

Single  Integrative  Process.  Our  report  recommended  a  single  integrative 
acquisition  process  within  a  JTF  to  allow  an  enhanced  use  of  acquisition  teams 
in-theater.  This  would  assist  in  creating  a  critical  mass  of  acquisition  expertise, 
thus  allowing  the  COCOM  and  JTF  commander  strategic  unity  and  flexibility 
with  respect  to  its  contract  support.  With  dispersed  contracting  organizations 
in  Iraq,  acquisition  personnel  were  hard-pressed  to  devote  time  to  strategic 
thinking  due  to  a  focus  on  daily  tactical  considerations.  Functions  such  as 
resource  allocation,  balancing  skill  sets,  program  integration,  and  requirements 
prioritization  suffered.  As  an  example,  U.S.  Central  Command  (CENTCOM) 
replaced  a  task  on  the  Air  Force  Contract  Augmentation  Program  (AFCAP) 
with  a  theater-wide  air  traffic  services  contract.  In  Iraq,  however,  a  different 
contract  provided  the  service  (D.  A.  Scott,  personal  communication,  January 
4,  2006),  thus  setting  the  stage  for  coordination  issues  that  would  have  been 
simpler  had  a  single  contract  been  awarded.  GAO  also  recognized  these 
problems  as  stemming  from  a  lack  of  coordination  (GAO,  2005).  We  proposed 
a  joint  contracting  activity  provide  a  single  integrative  acquisition  process  to 
evaluate  these  disparities  and  provide  the  best  acquisition  strategy  possible. 
This  recommendation  is  a  core  tenet  of  JP  4-10. 

An  integrated  approach  would  allow  the  joint  contracting  activity  to  pool 
resources  and  optimize  acquisition  decisions  at  critical  points  as  required  by 
the  JTF  Commander.  This  would  present  the  opportunity  to  leverage  specific 
skill  sets  to  fulfill  high-priority  acquisition  activities.  In  an  interview  with  Maj  Gen 
Darryl  A.  Scott,  USAF,  Joint  Contracting  Command’s  commander  in  Iraq,  he 
stated,  “There  are  other  assets  in-theater  that  could  be  used  to  balance  workload 
to  make  sure  high-priority  and/or  high-payoff  work  gets  addressed”  (D.  A.  Scott, 
personal  communication,  January  4,  2006).  Former  JCC  Iraq  Commander  BG 
Steven  Seay,  USA  (Ret.),  also  acknowledged  this  deficiency  and  recommended 
acquisition  personnel,  including  contracting  officers  and  other  specialists,  be 
consolidated  into  a  single  organization  (S.  M.  Seay,  personal  communication, 
January  24,  2006).  The  annexes  to  our  report  recommended  three  integrated 
organizational  constructs— one  each  for  large,  medium,  and  small  task  forces. 
Also  in  its  annexes,  JP  4-10  recommends  three  constructs:  Service  Component 
support  to  its  own  forces,  a  "Lead  Service  Theater  Support  Contracting 
Organization,”  and  a  “Joint  Theater  Support  Contracting  Organization.” 

Title  10  Authorities.  Our  report  recommended  the  Services  use  their  inherent  Title 
10  authorities  to  ensure  resources  and  contracting  authorities  would  be  in  place. 
Chapter  2,  paragraph  7  of  JP  4-10  describes  the  Service  Title  10  authorities  and 
how  to  use  those  authorities  to  enhance  joint  contracting  activity. 
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Coordination  Council.  Our  report  recommended  creation  of  a  Coordination  Council 
to  enhance  resource  cooperation  among  the  Services.  Roughly  analogous  to 
the  Coordination  Council,  JP  4-10  directs  the  establishment  of  the  Combatant 
Commander  Logistics  Procurement  Support  Board  (CLPSB)  to  deal  with  general 
policies  and  AOR-wide  issues  related  to  contracting  support. 


REQUIREMENTS  GENERATION/PRIORITIZATION  SUPPORT 

We  recommended  in  2006,  the  JTF  contracting  command  entity  should 
have  the  authority  to  operate  an  acquisition  review  board  on  behalf  of  the  JTF 
commander  to  collect  and  prioritize  contracting  requirements.  Traditionally, 
a  contracting  or  acquisition  activity  does  not  generate  requirements.  As 
happened  in  Iraq,  however,  the  requiring  activities,  especially  those  responsible 
for  stability  and  reconstruction,  did  not  always  have  sufficient  resources  or  time 
to  integrate  and  prioritize  requirements  across  the  theater.  Iraq’s  JCC  filled  in 
much  of  the  gap  with  a  relative  degree  of  success.  Therefore,  a  review  board 
should  have  the  flexibility  to  perform  these  functions  as  needed  and  provide 
recommendations  to  the  JTF  Commander  or  other  supported  customers.  These 
recommendations  would  consider  declared  needs  from  the  operators.  Upon 
receipt  of  the  customer’s  direction,  the  board  would  integrate  requirements 
appropriately,  develop  acquisition  strategies,  and  execute  contracts. 

JP  4-10  directs  the  creation  of  a  Joint  Acquisition  Review  Board  (JARB) 
specifically  to  control  requirements  generation  and  prioritization.  JP  4-10  also 
directs  creation  of  a  Joint  Contracting  Support  Board  (JCSB)  for  the  purposes 
of  assigning  prioritized  requirements  to  the  best  contracting  activity.  Our 
construct  assigned  this  to  the  joint  contracting  authority  without  identifying 
a  separate  board  to  perform  this  task.  In  addition,  our  research  supported  the 
need  to  assign  tasks  to  contracting  entities,  regardless  of  Service,  to  ensure 
balanced  workload  and  matched  skill-sets/workforce  availability.  JP  4-10 
agrees,  and  in  fact  offers  a  better  and  more  refined  approach  to  our  original 
recommendation  in  2006. 

Our  proposed  doctrine  considered  accommodation  of  coalition  forces 
and  interagency  support  for  contingency  and  post-contingency  (Phase  IV) 
operations.  JP  4-10  addresses  interagency  and  coalition  contracting  needs 
extensively.  Interagency  and  coalition  support  is  a  core  tenet  of  JP  4-10. 

Our  study  recommended  contracting  be  embedded  in  COCOM  planning 
with  pre-designated,  trained  personnel  deploying  with  a  JTF  to  enable  the 
combatant  commanders  to  execute  their  acquisition  missions  effectively  and 
efficiently  (GAO,  2003).  This  is  again  a  core  tenet  of  JP  4-10.  The  document 
addresses  COCOM  and  JTF  planning  considerations  extensively. 

Our  doctrine  proposal  recommended  COCOM  authorities  define  unique 
roles  and  responsibilities  to  match  the  planning  requirements  for  each  JTF.  We 
recommended  the  COCOM  consider  subordination  of  a  contracting  command 
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entity  to  the  JTF  leadership  and  OPCON  of  in-theater  acquisition  resources 
to  the  contracting  command  entity  to  improve  insight,  directive  authority, 
integration,  and  adherence  to  strategic  policies.  JP  4-10  describes  a  similar 
position  with  directive  roles  and  responsibilities,  while  leaving  enough  flexibility 
for  the  COCOMs  to  plan  to  their  unique  needs. 


CONTINGENCY  TO  SUSTAINMENT 

Our  report,  as  part  of  planning  strategy,  recommended  contracting 
activities  transition  from  higher  risk  (contingency  combat  operations)  to  lower 
risk  (sustainment)  at  appropriate  times  for  the  mission.  In  this  transition,  the 
contract  type  used  should  shift  from  cost  plus  (more  risk  on  the  government 
where  speed  and  quality  is  critical)  to  fixed  price  (less  risk  on  the  government 
where  cost  efficiency  grows  in  importance).  When  a  contractor  does  not  have 
highly  variable  costs  associated  with  security  protection,  the  U.S.  Government 
has  a  healthier  environment  to  competitively  award  firm  fixed  price  (FFP) 
contracts  and  make  better  use  of  funding.  In-theater  acquisition  expertise 
can  best  decide  when  to  change  the  type  of  contract.  For  example,  one  of  the 
reasons  the  Civilian  Augmentation  Program  (CAP)  contracts  were  so  expensive 
was  they  were  all  flexibly  priced— an  appropriate  view  with  a  highly  fluid 
operational  environment.  However,  as  parts  of  an  operation  stabilize,  fixed  price 
contracts  generate  better  value  (D.  Scott,  personal  communication,  2006).  JP 
4-10  specifically  mentions  this  strategy  and  identifies  potential  crossover  points 
for  changing  contract  strategies. 

This  concept,  in  our  view,  is  critically  important.  The  transition  from 
contingency  to  sustainment  contracting  is  necessary  to  improve  the  cost 
efficiency  of  operations  over  time.  Joint  Contracting  Activity  leadership  can 
help  provide  JTF  commanders  appropriate  strategies  for  the  transition.  At  a 
minimum,  a  general  transition  concept  added  to  deliberate  planning  will  help 
with  execution.  The  inherent  flexibility  offered  by  a  CAP  instrument  comes  at  a 
premium;  and  at  some  point,  the  cost  of  that  flexibility  may  not  be  necessary. 
In  addition,  commodity  groups  (water,  food,  construction  materials,  depot 
maintenance)  may  transition  at  different  times  and  under  different  local 
conditions.  Transition  planning  should  allow  for  greater  competition— a  critical 
step  as  risks  mitigate  and  cost  becomes  a  greater  consideration.  JP  4-10, 
Chapter  8.  Section  C  (3)  specifically  describes  these  considerations. 


FAR/DFARS  MODIFICATIONS 

Our  report  recommended  the  Federal  Acquisition  Regulation  (FAR)  and 
Defense  Federal  Acquisition  Regulation  Supplement  (DFARS)  be  reviewed  and 
adjusted  to  better  serve  contingency  and  post-contingency  conditions.  For 
example,  the  security  environment  in  Iraq  drove  many  contracting  officers  to 
write  cost  contracts  for  Operations  and  Maintenance  (O&M)  funding.  These 
contracts,  however,  typically  required  the  U.S.  Government  to  take  ownership 
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of  the  material  and/or  facilities  used  by  the  vendor  after  the  contract  closed 
out.  As  physical  goods  accrued  from  the  conduct  of  these  contracts,  property 
management  became  a  substantial  burden.  Additionally,  much  of  the  equipment 
purchased  was  of  limited  value  to  the  government.  We  recommended 
regulations  consider  more  flexibility  in  funding  thresholds,  property  purchase 
requirements,  and  solicitation  timelines  to  eliminate  or  minimize  these  problems 
CD.  A.  Scott,  personal  communication,  January  4,  2006).  Others  we  interviewed 
supported  the  contention  that  the  FAR  and  DFARS  should  either  be  changed 
or  supplemented  to  ensure  proper  contingency  guidance  and  authorities 
(C.  D.  Blake,  personal  communication,  January  19,  2006,  &  A.  B.  Bell,  personal 
communication,  February  8,  2006).  As  such,  we  recommended  in  our  report 
that  each  COCOM  obtain  specific  advance  regulation  waivers  for  each  plan 
as  part  of  the  planning  process.  These  would  activate  automatically  upon 
plan  implementation.  JP  4-10  chose  not  to  give  this  authority  to  the  COCOMs. 
However,  it  did  place  this  authority  with  the  Director,  Defense  Procurement  and 
Acquisition  Policy— a  position  reporting  to  the  Office  of  the  Under  Secretary 
of  Defense  for  Acquisition,  Technology,  and  Logistics.  This  is  where  authority 
to  change  DFARS  for  other  tasks  resides  already,  so  the  choice  was  obvious. 
JP  4-10  specifically  enhances  JTF  operations  by  assigning  this  DFARS  mod 
responsibility  for  operational  contracting  directly  and  clearly  to  this  director. 


OPERATIONAL  AND  STRATEGIC  TRAINING 

We  recommended  the  doctrine  establish  training  and  exercise  requirements 
to  generate  mature  in-theater  acquisition  capacity,  capable  of  meeting  mission 
needs  while  operating  in  austere  environments.  We  stated  the  doctrine  should 
require  that  COCOMs  consider  broad  skill  sets  for  a  joint  contracting  activity 
including  contracting,  program  management,  financial,  legal,  quality  assurance, 
and  information  technology  to  meet  contract  management  requirements 
in  the  AOR.  The  doctrine  should  call  for  “train  the  trainer”  functions  to  aid 
in  planned  transition  to  host  countries  and  identify  who  should  pay  for  the 
required  exercises.  Appendix  G  of  JP  4-10  describes  broader  skill  sets  and  their 
importance,  especially  in  the  organizational  construct  for  large  JTF  operations. 

Our  research  led  us  to  recommend  COCOM  and  JTF  staff  acquisition 
training  to  help  them  understand  how  to  best  use  acquisition  capabilities  for 
military  and  political  objectives.  For  example,  with  a  joint  contracting  activity 
providing  unity  of  effort,  acquisition  capacity  can  be  re-directed  temporarily 
to  meet  higher  JTF  priorities.  As  such,  an  Air  Force  (AF)  contracting  officer 
could  be  tasked  to  contract  for  line  haul  to  get  ammo  up  to  a  heavily  engaged 
Marine  unit.  The  following  month,  when  the  AF  needs  contracting  capacity  to 
build  an  airbase,  Army  resources  could  temporarily  augment  AF  contracting 
officers  to  support  the  requirement.  Doctrine  should  specifically  identify  the 
organization  responsible  for  conducting  these  training  programs.  To  support 
political  objectives,  a  JTF  could  improve  a  theater-wide  strategy  of  contracting 
with  local  companies,  thus  putting  local  personnel  to  work.  Joint  Publication 
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4-10  provides  extensive  detail  defining  terms,  describes  contracting  command 
authorities  and  structure  (which  is  different  than  operational  command 
structure),  and  offers  the  value  and  costs  of  contractor  support  in  the  AOR.  In 
essence,  JP  4-10  is  the  capstone-training  document  for  COCOM  and  JTF  staffs 
in  the  contracting  area.  As  such,  JP  4-10  fully  incorporates  our  report’s  overall 
training  recommendations. 


IMPROVED  SUPPORTING  AGENCY  ASSISTANCE 

We  recommended  use  of  an  in-theater  contracting  command  entity  to 
improve  supporting  agency  assistance  to  the  JTF.  As  an  example,  for  Operation 
Iraqi  Freedom  (OIF),  the  Defense  Contract  Management  Agency  (DCMA)  was 
administering  a  Stryker  repair  contract  in  Qatar  that  was  generally  meeting 
contract  performance  requirements.  Meanwhile  another  contractor’s  up- 
armoring  efforts  were  behind  schedule,  but  DCMA  had  not  received  a  delegation 
to  work  those  issues  (D.  A.  Scott,  personal  communication,  January  4,  2006). 
DCMA’s  core  mission  is  to  help  the  DoD  better  manage  contracts.  In  fact,  DCMA 
did  not  receive  delegations  to  several  of  the  most  challenging  contracts,  either 
at  a  quality  assurance  level  or  more  extensively.  A  unified  and  coordinated 
effort  expressed  through  a  centralized  contracting  activity  could  better  direct 
supporting  agency  assistance  when  and  where  needed.  Adding  contracting 
requirements  into  deliberate  planning  will  force  the  COCOM  to  consider  the 
best  use  of  supporting  agencies.  The  GAO  had  also  recognized  the  lack  of 
coordination  as  a  systemic  problem  since  the  late  1980s  (Waxman  &  Dingell, 
2004,  p.  5).  JP  4-10  correctly  defines  many  supporting  agency  missions  and 
how  they  can  specifically  support  COCOM  and  JTF  commanders. 


CARE  AND  FEEDING  OF  CONTRACTORS  ON  THE  BATTLEFIELD 

Contractors  perform  critical  functions  for  JTFs  but  also  need  support. 
During  OIF  and  other  contingencies,  contractor  support  requirements 
(housing,  food,  security,  etc.)  were  not  uniform,  due  at  least  in  part  because  the 
contracts  awarded  from  various  agencies  across  the  DoD  did  not  benefit  from 
authoritative  COCOM  guidance.  The  sheer  number  of  contractors  and  various 
contracting  instruments  made  it  “virtually  impossible  to  keep  track  of  who  eats 
for  free  and  who  must  pay”  based  on  the  terms  and  conditions  set  forth  in  the 
contracts  (D.  A.  Scott,  personal  communication,  January  4,  2006).  The  DoD’s 
response  to  this  problem  was  to  create  an  expensive  and  complex  system  of 
control.  Unfortunately,  as  has  happened  in  many  cases  in  Iraq,  this  did  not  fully 
solve  the  problem  and  did  not  prevent  many  contractors  from  receiving  services 
to  which  they  were  otherwise  not  entitled.  With  insufficient  control  of  the 
contractors,  some  companies  underperformed  by  using  government  support 
without  reimbursement,  whereas  their  contract  required  the  vendor  to  pay  for 
or  separately  provide  those  services.  This  created  funding  inefficiencies.  JP  4-10 
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devotes  much  discussion  to  this  topic  and  directs  the  creation  of  a  Contractor 
Support  Integration  Plan  (CSIP)  designed  specifically  to  fix  these  kinds  of  issues. 

This  type  of  COCOM  plan  is  also  critically  important  in  using  CAP  contracts. 
For  example,  in  the  case  with  Iraq,  Procurement  Contracting  Officers  (PCO)  for 
LOGCAP  did  not  have  sufficient  guidance  to  determine  how  many  contractors 
needed  support,  what  type  of  support  they  needed,  and  whether  it  would  be 
reimbursable  (GAO,  2004b).  This  left  CENTCOM,  in  the  case  of  OIF,  unable  to 
plan  LOGCAP  support  effectively  because  the  huge  number  of  contractors 
sent  and  specific  status  of  each  contract  could  not  be  determined  on  any  given 
day.  Without  this  information,  the  number  of  beds,  meals,  and  support  services 
needed  was  a  difficult  target  to  identify  (D.  A.  Scott,  personal  communication, 
January  4,  2006;  C.  M.  Bolton,  personal  communication,  January  19,  2006;  &  S. 
M.  Seay,  personal  communication,  January  24,  2006).  Again,  JP  4-10  devotes  a 
considerable  amount  of  detail  in  describing  how  each  of  the  Services  runs  their 
LOGCAP  programs,  thus  laying  the  groundwork  for  averting  or  minimizing  cost 
inefficiencies  in  this  arena. 


SUPPORT  FOR  A  JOINT  CONTRACTING  ACTIVITY 

In  interviewing  19  personnel  for  our  study,  we  found  the  following 
consistent  views  as  shown  in  the  appendix.  Significantly,  while  differences  in 
recommendations  for  contracting  structure  existed  between  the  interviewees, 
most  agreed  on  the  need  for  joint  doctrine,  proper  resourcing,  and  a  central 
organization  to  control  contract  management  in  theater.  Most  also  agreed 
planning  and  training  were  key  considerations.  Several  of  the  interviewees 
also  recommended  LOGCAP  and  other  contingency  contracts  be  included  in 
operations  plans  (C.  M.  Bolton,  personal  communication,  January  19,  2006,  &  S. 
M.  Seay,  personal  communication,  January  24,  2006). 


HEAD  OF  CONTRACTING  AUTHORITY  AND  WARRANTS 

In  our  study,  we  recommended  Head  of  Contracting  Authority  (HCA)  be 
resident  in-theater  in  the  Joint  Contracting  Activity.  We  also  recommended 
as  a  best  approach  the  issuing  of  warrants  within  theater.  We  find  consistent 
recommendations,  also  in  JP  4-10,  which  designate  the  HCA  should  reside 
in-theater.  It  also  directs  Senior  Contracting  Official  assignment  to  the 
Joint  Contracting  Activity  to  issue  warrants,  also  known  as  Certificates  of 
Appointment,  in-theater  efficiently  under  a  single  policy. 


CONTRACTING  OFFICER  OPCON 

We  also  recommended,  for  unity  of  effort,  all  contracting  personnel  should 
be  OPCON  to  the  Joint  Contracting  Activity,  although  the  affected  personnel 
do  not  need  to  reside  in  a  single  location.  Initially,  when  effectiveness  is  most 
critical,  acquisition  professionals  need  the  latitude  to  operate  co-resident  with 
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their  units  and  other  agencies  to  meet  mission  requirements.  This  approach 
is  less  efficient  but  is  more  effective  at  supporting  highly  variable  operations, 
the  kind  of  operations  most  likely  found  at  the  beginning  of  contingency 
operations.  OPCON  relationships  to  a  central  authority  are  still  possible  in  this 
environment,  and  will  allow  better  flow  of  strategic-level  acquisition  advice, 
including  the  identification  of  existing  and  available  contracting  instruments. 
To  support  this,  we  specifically  recommended  using  centralized  control  and 
decentralized  execution  for  contracting,  where  guidance  would  come  from  a 
centralized  authority  but  contracting  officers  would  travel  with  their  units  and 
directly  support  their  lower  echelon  commanders.  The  key  to  this  strategy  is 
the  concept  of  centralized  control  and  decentralized  execution  of  contracting 
activity  (Houglan,  2006,  p.  25).  The  JP  4-10  does  in  fact  use  the  same  concept 
of  centralized  control  and  decentralized  execution  with  an  OPCON  relationship 
to  a  centralized  contracting  authority. 


Conclusions 

History  has  shown  the  increasing  importance  of  contractors  on  the 
battlefield.  The  scope  and  depth  of  their  support  made  the  integration  of  DoD 
and  contractor  operations  essential  for  mission  success.  Prior  to  2008,  joint 
doctrine  did  not  exist  to  codify  the  necessary  lessons  learned  from  previous 
experiences.  Now  with  the  publication  of  Joint  Publication  4-10,  Operational 
Contract  Support,  integration  of  contractors  on  the  battlefield  is  squarely 
in  joint  doctrine  with  sufficient  detail  to  prevent  relearning  lessons  from 
previous  operations. 

Our  report,  written  in  2006  for  the  Industrial  College  of  the  Armed  Forces; 
research  from  many  others;  and  a  high  measure  of  momentum  and  desire  on  the 
part  of  the  Joint  Staff  and  the  Services  were  significant  catalysts,  culminating 
in  the  publication  of  JP  4-10.  Twenty-four  of  the  26  recommendations  from  our 
2006  report  were  incorporated  into  JP  4-10— some  almost  verbatim  and  others 
in  highly  variant  versions  of  what  we  originally  recommended. 

Publication  of  the  JP-4-10  represents  diligent  work  on  the  part  of  the  Joint 
Staff,  going  far  beyond  the  parameters  of  our  original  report  and  providing 
much-needed  depth  and  breadth  to  operational  contracting  considerations. 

Finally,  the  privatization  of  some  aspects  of  acquisition  and  contracting 
during  warfare  has  led  to  many  interesting  and  complex  issues.  For  that  reason, 
we  want  to  encourage  each  of  you,  especially  those  who  are  not  contracting 
officers  and  expect  to  deploy  to  a  JTF/contingency  location,  to  review  JP  4-10 
in  its  entirety. 
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APPENDIX 

LEVEL  OF  INTERVIEWEES’  SUPPORT  FOR  THE  CONCEPT  OF  A  JOINT 
CONTRACTING  ACTIVITY 
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Interviewee 

o 

& 
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Reference 

Mr.  Claude  M.  Bolton 

X 

X 

X 

X 

January  19,  2006 

LTG  William  E.  Mortenson,  USA 

X 

X 

X 

X 

January  25,  2006 

LTG  David  H.  Petraeus,  USA 

X 

X 

X 

X 

January  4,  2006 

Mr.  Lee  H.  Thompson 

X 

X 

X 

X 

February  9,  2006 

Maj  Gen  Darryl  A.  Scott,  USAF 

X 

X 

X 

X 

January  4,  2006 

MG  John  M.  Urias,  USA 

X 

X 

X 

X 

October  24,  2005 
(Briefing) 

BG  Steve  M.  Seay,  USA 

X 

X 

X 

X 

January  24,  2006 

RADM  Marty  J.  Brown,  USN 

X 

X 

X 

X 

March  1,  2006 

Col  Anthony  B.  Bell,  USAF 

X 

X 

February  8,  2006 

Col  Casey  D.  Blake,  USAF 

X 

X 

X 

January  19,  2006 

CDR  William  F.  Reich,  USN 

X 

X 

February  10,  2006 

CDR  Forest.  W.  Browne,  USN 

X 

X 

X 

January  25,  2006 

LtCol  Stephen  Elliot,  USMC 

X 

X 

February  3,  2006 

LtCol  John  E.  Cannady,  USMC 

X 

X 

X 

X 

January  23,  2006 

LtCol  Mark  A.  Hobson,  USMC 

X 

X 

January  18,  2006 

CDR  Robert  C.  Cox,  USN 

X 

X 

X 

February  2,  2006 

COL  Ainsworth  B.  Mills,  USA 

X 

X 

X 

X 

January  19,  2006 

COL  Stephen  B.  Leisenring,  USA 

X 

X 

X 

X 

February  9,  2006 

COL  Jacob  B.  Hansen,  USA 

X 

X 

X 

X 

February  6,  2006 

Note:  An  "x”  in  any  of  the  blocks  signifies  the  interviewee  specifically  endorsed  the 
need  for  Joint  Doctrine,  strategic  planning,  coordinated  resourcing,  and/or  improved 
training.  Significantly,  most  of  the  personnel  interviewed  for  this  article  recommended 
planning  to  ensure  proper  allocation  of  resources  to  top  JTF  priorities.  This  is  yet 
again  a  strong  sign  that  practitioners  in  the  field  universally  desired  joint  contracting 
operations  doctrine.  The  dear  recognition  of  the  need  for  doctrine  among  contracting 
practitioners  was  undeniably  helpful  in  supporting  the  creation  of  JP  4-10.  The  desire 
and  momentum  was  certainly  in  place  from  2006  to  2008,  and  the  DoD  took  advantage 
of  that  momentum  to  create  JP  4-10. 
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APPLICATION  OF 
SYSTEMS  ENGINEERING 
TO  RAPID  PROTOTYPING 
FOR  CLOSE  AIR  SUPPORT 

f  John  M.  Colombi  and  Richard  G.  Cobb 

Twenty-first  century  military  operations  have  brought  forth 
many  new  challenges  for  the  Armed  Forces  of  the  United 
States.  One  such  challenge  is  with  new  operating  environ¬ 
ments,  where  current  systems  are  not  always  effective.  While 
it  is  desirable  to  apply  a  systems  engineering  approach  to  best 
meet  critical  user  needs,  there  may  be  a  misconception  that 
systems  engineering  requires  a  lengthy  and  detailed  process 
not  nimble  enough  for  a  rapid  prototyping  effort.  This  article 
describes  how  a  classic  systems  engineering  methodology 
was  successfully  tailored  to  the  rapid  development  of  poten¬ 
tial  material  solutions  to  meet  a  critical  operational  need.  Key 
observations  are  drawn  from  this  experience  and  formulated 
into  heuristics  for  tailoring  systems  engineering  for  future 
rapid  prototyping  efforts. 


Keywords:  Systems  Engineering,  Prototyping,  Rapid 
Product  Development,  Project  Selection,  Close  Air 
Support 
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Within  the  U.S.  Air  Force,  a  critical  need  has  emerged  for  an  added  capability 
associated  with  the  position  of  Joint  Terminal  Attack  Controller  (JTAC)— the 
Air  Force  airman  trained  to  interface  with  aircraft  to  request  and  direct  Close 
Air  Support  (CAS)  attacks:  to  quickly  pinpoint  the  location  of  friendly  ground 
forces  and  communicate  their  location  to  CAS  aircraft.  Current  operations  in 
urban  environments  have  placed  JTACs  in  very  close  proximity  to  enemy  forces 
and  reduced  the  time  to  communicate  with  CAS  assets.  This  close  proximity 
and  time  compression,  coupled  with  the  complexity  of  the  urban  terrain,  has 
made  it  difficult  for  the  JTAC  to  direct  an  air  attack  using  current  systems  and 
tactics  while  maintaining  an  acceptable  fratricide  risk.  Thus,  a  Friendly  Marking 
Device  (FMD)  that  allows  a  JTAC  to  quickly  and  accurately  identify  the  position 
of  friendly  ground  personnel  to  CAS  aircraft  has  emerged  as  a  critical  need. 


CAN  A  DEVELOPMENT  EFFORT  BE  RESPONSIVE  ENOUGH 
TO  REACT  TO  CRITICAL  NEEDS  WHILE  STILL  BENEFITING 
FROM  THE  RIGOR  OF  SYSTEMS  ENGINEERING? 


Systems  engineering  offers  a  rigorous  and  repeatable  methodology 
for  translating  a  critical  need  into  a  viable  solution  (Defense  Acquisition 
University  [DAU],  2001).  However,  the  perception  that  it  necessitates  a  lengthy 
and  detailed  process  may  contribute  to  a  misconception  that  the  benefits  of 
systems  engineering  must  be  traded  off  to  be  able  to  respond  quickly  to  critical 
user  needs.  This  perception/misconception  juxtaposes  a  key  question:  Can  a 
development  effort  be  responsive  enough  to  react  to  critical  needs  while  still 
benefiting  from  the  rigor  of  systems  engineering? 

This  article  attempts  to  answer  that  question  by  detailing  an  effort  to  tailor 
and  apply  systems  engineering  to  a  collaborative  research  project  to  rapidly 
prototype  novel  designs  for  the  FMD.  It  describes  the  methods  employed 
and  offers  key  observations  from  the  experience  as  lessons  learned.  From  the 
lessons,  heuristics  are  derived  to  guide  the  tailoring  and  application  of  systems 
engineering  to  future  rapid  prototyping  efforts. 

The  JTAC  user  identified  the  critical  need  for  a  new  way  to  mark  the 
location  of  friendly  ground  forces.  Under  the  auspices  of  the  Air  Force  Research 
Laboratory  (AFRL)  Rapid  Reaction  Program— a  program  designed  to  match 
innovative  research  initiatives  to  critical  needs— an  effort  began  aimed  at 
identifying  and  applying  technology  to  the  critical  operational  need,  and 
resulting  in  the  generation  of  a  viable  solution. 
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Method 


PROJECT  DEFINITION 

The  first  step  in  defining  the  project  was  to  assemble  a  core  project  team 
to  guide  the  development  effort.  During  this  step,  key  stakeholders  were 
identified— user/customer,  project  sponsor,  systems  engineers,  and  technology 
experts.  The  core  team  then  worked  to  understand  the  operational  need 
and,  thereby,  define  the  objective  of  the  project:  Develop,  demonstrate,  and 
transition  a  marking  solution  that  enables  a  JTAC  to  establish  a  common  point- 
of-reference  with  a  CAS  asset  such  that  the  CAS  asset  can  attack  an  intended 
target  while  avoiding  fratricide. 

Constraining  factors  such  as  cost,  schedule,  technology  maturity,  resource 
availability,  and  operational  limitations  were  clearly  identified.  Arguably, 
the  most  significant  constraint  on  the  project  was  a  compressed  schedule, 
inherent  to  the  rapid  reaction  process.  Driven  by  the  desire  to  rapidly  field  a 
prototype,  the  project  was  constrained  to  5  months.  These  constraints  became 
fundamental  elements  driving  several  key  evaluation  and  technical  focus  factors 
in  our  systems  engineering  process. 


TAILORED  APPROACH 

After  careful  consideration  of  a  variety  of  approaches,  the  classic  Vee 
model  described  in  Dennis  M.  Buede’s  (2000)  text  was  tailored  and  selected 
as  the  basis  for  this  project.  Both  the  construct  and  execution  of  the  model 
were  modified  to  accommodate  the  constraints  identified  at  the  outset.  The 
tailored  Vee  model  (Figure  1)  follows  the  general  construct  of  the  classic  Vee 
model  in  that  requirements  solicitation  and  definition  occurs  down  the  left 

FIGURE  1.  TAILORED  VEE  MODEL 
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TABLE  1.  SAMPLE  USE  CASE 


Urban  Close  Air  Support  Use  Case 

Use  Case  Name 

Name:  Urban  Close  Air  Support 

Brief  Description:  Describes  the  process  directing  a 

CAS  attack  in  an  urban  environment. 

Actors  Involved 

Joint  Tactical  Air  Controller  (JTAC):  A  certified 

servicemember  who  directs  the  action  of  aircraft 

engaged  in  close  air  support. 

•  Goal— Accurately  identify  target  and 
friendly  forces  to  CAS  aircraft. 

Close  Air  Support  (CAS)  Aircraft:  Aircraft  tasked  to 
support  close  air  support  operations. 

•  Goal— Accurately  acquire  target  and 

friendly  position. 

Air  Support  Operations  Center  (ASOC):  The  principal 
air  control  agency. 

Preconditions 

JTAC  has  communication  with  ASOC. 

JTAC  has  requested  CAS  support. 

CAS  aircraft  tasked  to  support  the  JTAC. 

CAS  has  aircraft  in  contact  with  JTAC. 

Success  Guarantee 

CAS  aircraft  provide  bombs  on  target. 

There  is  no  fratricide  of  friendly  forces. 

Collateral  damage  has  been  minimized. 

Flow  of  Events 

Main  Success  Scenario:  Sequential,  numbered  steps 
to  carry  out  the  task. 

Postconditions 

CAS  Aircraft:  Provide  bombs  on  target. 

side  (decomposition  and  definition),  design  engineering  occurs  at  the  vertex, 
and  qualification  occurs  moving  up  the  right  side.  An  important  element 
of  tailoring  as  applied  herein  involves  the  recognition  that  the  output  of  this 
tailored  Vee  model  is  not  a  validated  system  ready  for  use  in  the  field.  Rather  it 
is  an  analytically  tested  and  evaluated  prototype  that  may  be  easily  readied  for 
production  and,  ultimately,  used  in  the  field. 


PROBLEM  DEFINITION 

To  state  the  problem  in  solution-independent  terms,  the  definition  process 
began  by  exploring  the  problem  domain.  After  a  literature  search  of  typical  CAS 
processes  (Joint  Chiefs  of  Staff,  2003;  Pirnie  et  al.,  2005),  a  set  of  elicitation 
questions  was  developed  to  help  define  a  common  understanding  of  the  problem 
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FIGURE  2.  EXTERNAL  SYSTEMS  DIAGRAM 
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with  the  user.  These  questions  were  then  used  as  a  basis  for  interviewing  the 
user  representative  to  build  a  definition  of  the  problem. 

It  became  evident  the  original  problem  statement  did  not  capture  another 
perspective  that  existed— that  of  the  CAS  pilot.  To  correct  this,  experienced 
CAS  pilots  were  interviewed  in  a  similar  fashion  to  explore  their  perspective 
of  the  problem.  After  compiling  the  results  of  the  interviews,  the  problem  was 
stated  as:  The  Joint  Terminal  Attack  Controller  (JTAC)  lacks  a  covert  means  to 
quickly  and  accurately  mark  the  location  of  friendly  forces. 


OPERATIONAL  CONCEPT 

The  next  step  was  development  of  the  concept  of  operation  for  the  solution— 
the  vision  of  how  the  user  might  employ  the  resultant  device.  Borrowing  from 
software  engineering  (Larman,  2004),  the  concept  of  a  use  case  was  employed 
to  create  a  description  of  the  sequenced  actions  that  the  user  would  likely  follow 
in  employing  the  FMD  (Cockburn,  2001).  Table  1  shows  a  simplified  version  of 
the  basic  use  case  for  directing  CAS  attacks  in  an  urban  environment.  (This  is 
not  a  complete  use  case  and  is  included  for  illustration  only.) 

Buede  (2000,  p.  144)  states,  “The  single  largest  issue  in  defining  a  new 
system  is  where  to  draw  the  system’s  boundaries.”  As  the  project  progressed,  the 
value  of  defining  and  documenting  the  system  boundary  became  evident,  and 
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the  External  Systems  Diagram  shown  in  Figure  2  was  developed.  Creating  the 
External  Systems  Diagram  helped  highlight  the  key  interaction  in  the  operational 
concept— the  use  of  the  FMD  to  establish  a  common  point  of  reference  between 
the  JTAC  and  the  CAS  pilot. 


Requirements 

With  the  appropriate  data  from  the  informal  interviews  of  the  user  and  other 
stakeholders  as  guidance,  the  system  requirements  were  derived  in  detail  from 
the  operational  concept.  Once  the  initial  set  of  requirements  was  identified, 
it  was  validated  with  the  user  and  other  stakeholders.  In  addition,  the  user 


TABLE  2.  USER  REQUIREMENTS 


User  Requirements  with  Weights 

Type 

Requirements 

Weights  (1-10) 

Environmental 

Weather  Limitations 

9 

Day/Night  Limitations 

10 

Physical 

Waterproof 

8 

Shockproof 

8 

Power  Source 

8 

Weight 

10 

Size  Dimensions 

10 

Operational 

Signal  Duration 

10 

(Signal) 

Signal  Covertness 

10 

Signal  Field  of  View 

7 

Signal  Range 

10 

Accuracy  Resolution 

10 

Signal  Spectrum 

10 

System  Compromise 

2 

Unique  Signal 

2 

Signal  Delay 

10 

Operational 

Ease  of  Use 

8 

(System) 

Modification  Required 

8 

Unique  Signal  Display 

2 

Acquisition 

Long-term  Unit  Cost 

5 

(Long-term) 

Product  Feasibility 

8 

Acquisition 

Estimated  Cost 

5 

(Short-term) 

Prototype  Availability 

7 

System  Maturity 

7 
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and  other  stakeholders  provided  weights  for  each  requirement  to  determine 
their  priority.  Table  2  shows  a  sample  of  the  system  requirements  (without  the 
associated  values,  but  with  user  weights). 


OBJECTIVES  HIERARCHY 

In  making  a  decision  or  evaluation,  the  development  of  a  value  model  (in 
this  case,  an  objectives  hierarchy)  enables  the  systematic  identification  and 
application  of  user  value  to  multiple  attributes  of  a  decision.  Following  the 
approach  described  by  Ralph  L.  Keeney  (1992),  a  set  of  appropriate  objectives 
was  identified.  Attributes  to  measure  the  degree  to  which  the  objectives  are  met 
were  also  developed.  Finally,  a  hierarchy  defining  the  relative  weighting  of  the 
objectives  was  created  (Figure  3). 

The  use  case  and  user-expressed  desires  and  constraints  served  as  inputs 
into  the  development  of  the  hierarchy.  The  objectives  were  developed  by 


FIGURE  3.  SAMPLE  OBJECTIVES  HIERARCHY 
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FIGURE  4.  SAMPLE  UTILITY  CURVE 

Detection  Range  Utility  Curve 

1 


Detection  Range  (nm) 


working  closely  with  the  user/customer.  Once  the  basic  hierarchy  had  been 
constructed,  the  user  was  solicited  for  the  relative  weightings  that  define  the 
value  or  importance  of  each  of  the  various  objectives.  Relative  weights  for 
applicable  objectives  were  also  solicited  from  the  CAS  pilots.  Utility  curves 
were  produced  based  upon  the  information  gleaned  during  the  development 
of  the  problem  definition  and  operational  concept.  Risk-neutral  utility  curves, 
also  described  in  Keeney,  were  used  in  the  assessment  of  value  for  each  of 
the  characteristics  of  the  hierarchy.  Figure  4  shows  an  example  of  the  utility 
values  for  signal  detection  range.  The  assignment  of  utility  values  and  the 
performance,  physical,  and  environmental  element  utility  curves  were  based 
upon  user  requirements. 

The  objective  hierarchy  was  continually  updated  throughout  the  FMD 
systems  engineering  process  as  candidate  technologies  matured  and  were 
tested.  It  served  as  the  primary  decision-making  tool  for  initial  candidate 
selection,  as  well  as  the  subsequent  testing  and  evaluation  to  designate 
candidates  for  transition  to  full  development. 


DEVELOP  VALIDATION/VERIFICATION  CRITERIA 

The  next  step  involved  developing  the  criteria  necessary  to  verify  the  poten¬ 
tial  solutions  against  the  derived  requirements,  and  further  validating  them 
against  the  user  need  or  mission  requirement.  The  problem  statement,  opera¬ 
tional  concept,  and  requirements  set  served  as  the  sources  for  these  criteria. 
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The  basic  approach  involved  breaking  the  problem  down  into  critical 
operational  issues  (COI).  Measures  of  effectiveness  (MOE)  were  then  developed 
for  each  COI  to  help  evaluate  whether  or  not  a  particular  candidate  was  able 
to  resolve  the  issue.  MOEs  were  then  broken  down  into  specific  measures  of 
performance  (MOP)  that  could  be  measured  to  verify  the  candidate  design 
(Roedler  &  Jones,  2005;  Sproles,  2000;  Sproles,  2001).  Great  care  was  taken  to 
state  these  criteria  in  solution-independent  terms  such  that  the  evaluation  did 
not  suggest  or  favor  a  particular  type  of  solution. 


CANDIDATE  IDENTIFICATION  AND  DEVELOPMENT 

The  process  of  identifying  candidate  technologies  began  with  a  meeting 
of  the  stakeholders  to  present  the  critical  need  and  the  resulting  operational 
problem.  The  technology  experts  were  then  given  the  operational  concept 
and  the  requirements  for  the  FMD,  and  asked  to  identify  novel  technology 
candidates  to  solve  the  operational  problem.  An  initial  set  of  15  candidate 
technologies  resulted. 

This  initial  set  of  candidates  was  evaluated  for  feasibility  using  the 
objectives  hierarchy.  This  initial  evaluation  helped  to  eliminate  non-viable 
candidates.  Based  upon  this  evaluation,  the  initial  set  of  15  was  pared  down 
to  10  promising  candidates.  Over  approximately  3  months,  the  technology 
experts  worked  in  parallel  to  further  research  and  develop  their  respective 
ideas  for  solving  the  problem. 


LAB  PROTOTYPE  TESTING 

Many  of  the  decisions  to  this  point  had  been  made  based  upon  predictions, 
analytical  calculations,  and  bench  tests— analyzing  only  portions  of  the  device 
without  testing  full  functionality.  It  was,  therefore,  necessary  to  verify  the 
prototypes  through  lab  testing— testing  the  full  functionality  of  the  device  without 
subjecting  it  to  a  realistic  operational  environment.  Since  the  prototypes  were 
completed  at  different  times,  lab  testing  occurred  throughout  the  development 
period  rather  than  during  a  specific  test  period. 

To  proceed  to  the  field  demonstration,  prototypes  were  required  to  have 
been  successfully  verified  against  the  requirements  via  the  lab  testing.  The 
results  of  the  lab  tests  were  fed  back  into  the  objectives  hierarchy,  and  the 
candidate  technologies  were  again  evaluated  against  the  objectives.  As  a 
result  of  the  verification  process,  eight  prototypes  were  selected  to  proceed 
to  field  demonstration. 


OPERATIONAL  PROTOTYPE  FIELD  DEMONSTRATION 

To  properly  scope  the  demonstration,  the  team  developed  and  coordinated 
a  test  plan,  which  outlined  the  roles  and  responsibilities  of  each  participant 


294 


A  Publication  of  the  Defense  Acquisition  University 


www.dau.mil 


and  the  major  test  objectives.  Test  and  Evaluation  Management  guidance  is 
well  documented  (DAU,  2005).  The  test  objectives  were  derived  from  the  user 
requirements  and  MOPs  discussed  previously.  In  addition,  aircraft  flight  profile 
descriptions  were  developed,  and  a  prioritized  test  point  matrix  was  created. 
Finally,  data  requirements  were  documented  to  enable  post-flight  analysis  of 
prototype  performance. 

The  candidate  prototypes  were  taken  to  the  Nellis  Air  Force  Base  test 
range  for  the  field  demonstration.  The  evaluations  were  conducted  by  Air  Force 
operational  test  agencies  representing  both  user  communities:  the  JTAC  ground 
controllers  and  the  combat  aircrews. 


Evaluation  of  Results 

The  team  collected  and  reviewed  the  recorded  data  from  the  aircraft 
to  determine  the  maximum  detection  for  each  device  as  well  as  to  evaluate 
the  quality  of  the  detection  display  as  seen  from  the  aircraft.  JTAC  usability 


Overall,  the  FMD  project  successfully  applied 

SYSTEMS  ENGINEERING  TO  TAKE  A  CRITICAL  USER  NEED 
AND  RAPIDLY  PRODUCE  VIABLE  PROTOTYPES  THAT 
COULD  BE  TRANSITIONED  TO  PRODUCTION. 


assessments  and  aircrew  comments  were  also  gathered  and  reviewed  in  order 
to  evaluate  the  performance  of  the  prototype  devices.  While  not  a  quantitative 
measure,  the  user  assessments  of  the  prototypes  at  this  early  stage  were  deemed 
critical  as  they  would  provide  the  direction  for  the  next  phase  of  development- 
producing  the  FMD.  That  is,  once  the  basic  technology  is  proven,  it  must  still 
be  designed  to  meet  users’  expectations  for  form,  fit,  and  function.  With  this  in 
mind,  a  review  was  conducted  on  the  user  assessments  of  each  device,  noting 
favorable  characteristics  as  well  as  highlighting  key  areas  of  concern  to  be 
addressed  in  the  next  iterations  of  the  development  process. 


PRIORITIZATION  AND  SELECTION  OF  OPTIONS 

The  results  of  the  field  demonstration  were  fed  back  into  the  objective 
hierarchy.  Coupling  the  updated  ranking  from  the  objective  hierarchy  analysis 
with  engineering  judgment  and  qualitative  user  feedback,  the  team  selected  one 
candidate  technology  that  met  all  of  the  objectives  and  held  the  greatest  promise 
of  being  developed  into  a  system  capable  of  meeting  the  needs  of  the  user. 

Overall,  the  FMD  project  successfully  applied  systems  engineering  to 
take  a  critical  user  need  and  rapidly  produce  viable  prototypes  that  could 
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be  transitioned  to  production.  During  the  course  of  the  efforts,  the  systems 
engineers  gained  valuable  insight  into  the  application  of  systems  engineering 
to  rapid  prototyping.  The  remainder  of  the  article  focuses  on  key  observations. 


Key  Observations  and  Results 

In  this  section,  key  observations  are  made  about  the  FMD  project.  In 
particular,  each  section  presents  a  lesson  learned  and  briefly  describes  the 
impact  the  finding  had  on  the  project. 


UNDERSTANDING  KEY  CONSTRAINTS 

Observation:  Explicitly  stating  and  understanding  key  constraints  helped  guide  team 
decision  making  and  brought  clarity  to  choices. 

Several  key  constraints  were  established  at  the  beginning  of  the  project.  By 
stating  the  constraints  explicitly  from  the  outset,  the  entire  team  was  focused  on 
the  same  goals.  This  shared  understanding  guided  decision  making  throughout 
the  project.  In  particular,  it  made  the  choice  between  alternatives  relatively  clear 
when  conducting  trade-offs  and  candidate  evaluations. 


UNDERSTANDING  THE  LARGER  CONTEXT 

Observation:  An  understanding  of  the  larger  context  helped  in  developing  a  tailored 
systems  engineering  model  and  provided  a  long-term  framework  for  the  project. 

Part  of  tailoring  the  systems  engineering  approach  involved  understanding 
the  bigger  context  in  which  this  specific  rapid  prototyping  effort  fit.  The 
programmatic  boundary  helped  communicate  scope  to  all  the  stakeholders, 
and  helped  in  day-to-day  systems  engineering  management.  Figure  5  places 
the  modified  Vee  model  of  Figure  1  into  the  larger  context  of  a  longer-term 
development  fielding  of  future  CAS  systems  acquisitions.  In  this  context,  the 
rapid  prototyping  Vee  model  represents  the  first  increment  of  the  FMD  rapid 
fielding  effort.  This  can  also  be  viewed  as  the  first  spiral  in  the  context  of  the 
systems  engineering  spiral  model  as  shown  in  Figure  6.  This  understanding 
helped  to  modify  the  classic  Vee  model  to  one  in  which  the  end  state  was  a 
demonstrated  and  validated  FMD  prototype.  This  prototype  then  provided  both 
the  input  to  the  next  spiral  — FMD  production  design— as  well  as  a  refined  and 
validated  set  of  user  requirements  that  can  serve  as  important  inputs  for  future 
CAS  systems  acquisitions. 

In  the  spiral  development  context  (Boehm  &  Hansen,  2001),  FMD  production 
design  continues  the  spiral,  resulting  in  a  production-ready  design  to  “fill  the  gap” 
in  capability.  After  user  evaluation  and  acceptance  of  the  production  design, 
the  FMD  production  and  fielding  spiral  ensues.  A  formal  systems  acquisition 
program  for  an  advanced  FMD  was  envisioned  as  the  next  spiral. 
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FIGURE  5.  FRIENDLY  MARKING  DEVICE  (FMD)  ACQUISITION  CONTEXT 
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BORROWING  FROM  OTHER  DISCIPLINES 

Observation:  Proven  techniques  from  software  engineering  were  applicable  in  a  rapid 
hardware  prototyping  effort. 

The  field  of  software  engineering  has,  through  many  years  of  evolution, 
developed  a  very  elegant  approach  to  tame  the  complexity  and  constant 
change  of  modern  software  development.  Whereas  the  waterfall  approach 
(Royce,  1970)  treated  the  requirements  definition,  design,  and  testing  as  distinct, 
sequential  steps,  modern  approaches  such  as  the  Rational  Unified  Process 
(RUP)  (Krutchen,  2000)  emphasize  evolutionary  development  in  iterations. 
The  FMD  project  applied  key  tenets  from  the  RUP  to  the  rapid  development  of 
hardware  prototypes. 

The  sequential  waterfall  approach  presumes  that  the  requirements  for  the 
system  can  be  known  with  a  high  degree  of  certainty  from  the  outset  and  that 
those  requirements  remain  relatively  static  during  the  development  process.  In 
a  rapid  prototyping  effort,  this  is  not  very  likely  to  be  the  case,  particularly  when 
the  user  may  not  know  what  is  within  the  realm  of  the  possible  given  the  current 
state  of  the  technology  and  the  key  constraining  factors. 

The  RUP,  in  contrast,  makes  no  such  presumption  and  relies  on  short 
development  steps  with  rapid  feedback  to  adapt  the  design  as  requirements 
are  clarified.  The  FMD  project  resembled  the  RUP  in  that  it  included  an 
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FIGURE  6.  FRIENDLY  MARKING  DEVICE  (FMD)  IN  SPIRAL  CONTEXT 
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initial  exploratory  phase  much  like  an  inception  iteration.  This  phase  lasted 
approximately  4  weeks.  It  included  the  initial  meetings  with  the  user  and  the 
entire  project  team.  Accomplishments  included  creating  the  operational  concept 
(vision),  collecting  the  user’s  initial  requirements,  and  defining  the  scope  of  the 
project.  In  addition,  the  initial  technology  exploration  was  used  to  check  the 
feasibility  of  the  novel  technology  ideas.  Based  upon  initial  design  ideas  and 
performance  estimations,  the  user  was  able  to  refine  the  requirements  and  help 
eliminate  some  technology  candidates  because  of  their  size,  weight,  or  power 
consumption.  The  result  was  the  initial  list  of  ten  candidates. 

The  rest  of  the  project  (as  of  this  writing)  was  much  like  the  elaboration  phase 
of  the  RUP.  The  ten  initial  candidates  were  built  into  functioning  prototypes.  As 
the  designers  completed  various  phases  of  their  fabrication  work,  more  was 
learned  about  each  of  the  candidates.  This  new  knowledge  was  rapidly  fed  back 
into  the  process  to  further  refine  requirements  and  guide  the  project. 

Timeboxing  was  also  effective  for  the  FMD  project.  Two  candidate 
technologies  were  not  mature  enough  to  proceed  to  the  field  demonstration. 
Rather  than  slip  the  date,  those  candidates  were  excluded  from  the  field 
demonstration  with  the  intent  to  continue  their  development  and  take  them  to 
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the  field  during  a  later  iteration.  In  the  interim,  feedback  from  poor  field  results 
for  candidates  with  similar  technology  (i.e.,  employing  a  similar  type  of  emitter) 
showed  that  one  of  the  immature  candidates  would  not  be  a  viable  solution. 
That  candidate  was  eliminated,  saving  both  time  and  money. 


SELECTING  AND  USING  TOOLS 

Observation:  Selection  of  tools  suited  to  the  tailored  systems  engineering  approach 
facilitated  the  decision-making  process. 

In  making  any  decision,  the  development  of  a  value  model  enables  the 
systematic  identification  and  application  of  user  value  to  multiple  attributes  of 
a  decision.  The  FMD  rapid  development  environment  required  a  decision  tool 
that  effectively  used  the  limited  candidate  attribute  information,  preserved 
design-independent  solutions,  did  not  impose  a  large  analytical  overhead,  and 
effectively  identified  the  most  viable  alternatives. 

Within  the  framework  of  the  objectives  hierarchy,  a  “living”  multi¬ 
attribute  decision  tool  was  created  by  revisiting  the  phases  as  new  and  refined 
information  was  obtained.  In  this  way,  any  new  information,  such  as  better 
performance  estimates  or  actual  test  results,  was  quickly  fed  back  into  the 
objectives  hierarchy  to  give  a  new  snapshot  of  the  solution  space  in  terms  of  the 
stakeholders’  objectives. 

Buede  (2000)  discusses  how  the  use  of  objectives  hierarchy  can  be  used 
throughout  the  systems  design  life  cycle  to  support  trade  studies.  Another 
somewhat  unique  application  of  the  tool  was  that  the  objectives  hierarchy  was 
used  not  only  throughout  the  design  process  (down  the  left  side  of  the  Vee 
model),  but  also  as  an  analysis  tool  during  the  prototype  evaluation  process 
(up  the  right  side  of  the  Vee  model)  as  well.  The  objectives  hierarchy  provided 
a  mechanism  to  integrate  actual  prototype  test  data  with  long-term  rapid 
production  unit  attributes  such  as  projected  weight,  dimensions,  etc.,  into 
a  single,  scoreable  measure  to  compare  alternatives.  Doing  so  ensured  that 
important  production  and  usability  issues  were  considered  (via  estimates  and 
predictions)  in  the  final  candidate  selection. 


DEVELOPING  IN  PARALLEL 

Observation:  Parallel  development  helped  reduce  the  overall  risk  of  the  project. 

Managing  risk  is  part  of  any  project.  Rapid  prototyping  is,  arguably,  itself  a 
form  of  risk  management  in  that  the  aim  is  to  explore  a  solution  space.  However, 
in  the  case  of  the  FMD  project,  the  rapid  prototyping  attempted  to  respond  to 
a  critical  operational  need.  In  this  light,  there  was  significant  incentive  to  ensure 
that  some  solution  was  identified  that  would  be  acceptable  to  the  user. 

From  the  outset  of  the  project,  the  team  sought  to  reduce  the  risk  that  no 
acceptable  solution  would  be  found.  A  classic  risk  mitigation  technique  when 
dealing  with  innovative  and  often  immature  technology  is  to  pursue  multiple 


Application  of  Systems  Engineering  to  Rapid  Prototyping  for  Close  Air  Support 


October  2009 


2  9  9 


parallel  paths  towards  the  same  goal.  This  approach  was  used  on  the  FMD 
project.  At  the  initial  evaluation,  rather  than  selecting  a  single  candidate  to 
build  and  test,  the  team  attempted  to  prototype  all  of  the  candidates  that  were 
predicted  to  meet  the  user  need  based  upon  the  estimates  and  performance 
calculations  supplied  for  the  first  iteration  of  the  objectives  hierarchy. 

Another  way  that  the  parallelism  helped  the  effort  was  that  lessons  learned 
by  one  of  the  parallel  tracks  could  be  fed  back  into  the  rest  of  the  tracks  to  help 
guide  and  refine  the  remaining  work.  For  example,  early  lab  tests  showed  that 
modulation  was  especially  helpful  in  making  a  signal  more  discernible  to  the 
observer.  This  information  was  then  incorporated  into  the  remaining  designs  to 
help  further  reduce  risk. 


MAINTAINING  RIGOR  IN  A  RAPID  REACTION  PROJECT 

Observation:  A  development  effort  can  be  responsive  to  critical  operational  needs  while 
maintaining  the  rigor  of  systems  engineering. 

Organizations  often  have  very  formalized  and  standardized  systems 
engineering  processes  for  product  development.  Within  the  DoD,  the  systems 
engineering  process  is  often  associated  with  a  series  of  documentation 
requirements  (formal  plans,  requirements,  etc.)  flowing  through  a  rather  large 
management  and  oversight  function,  coupled  with  a  very  di  recti  ve  series  of  formal 
reviews  (DAU,  2001;  Department  of  Defense,  1993).  However,  the  underlying 
principles  of  systems  engineering  are  present  in  the  DoD  process  (DeFoe, 
1993).  When  the  overhead  of  the  standard  formal  review  and  documentation 
requirements  is  reduced,  a  very  realistic  approach  to  conducting  rapid  and 
innovative  development  is  generated.  In  fact,  a  common  misperception  is  that 
the  DoD  imposes  a  specific  systems  engineering  process.  Rather,  the  Defense 
Acquisition  Guidebook  outlines  standard  industry  systems  engineering  models 
and  emphasizes  that  “models  usually  contain  guidance  for  tailoring,  which  is 
best  done  in  conjunction  with  a  risk  assessment  on  the  program  that  leads  the 
program  manager  to  determine  which  specific  processes  and  activities  are  vital 
to  the  program”  (DAU,  2009,  p.  12). 

Based  upon  the  results  of  the  FMD  project,  the  conclusion  is  drawn 
that  by  effectively  tailoring  the  application  of  classic  systems  engineering 
methodologies  to  the  problem  at  hand,  a  development  effort  can  be  responsive 
to  critical  operational  needs  while  maintaining  the  rigor  of  systems  engineering. 


HEURISTICS  DISCUSSION 

Rather  than  attempting  to  provide  a  recipe  for  tailoring  the  application  of 
systems  engineering  to  a  rapid  prototyping  effort,  this  section  presents  the 
lessons  learned  during  the  FMD  project  in  the  form  of  heuristics  that  can  help 
guide  similar  efforts  in  the  future  (Maier  &  Rechtin,  2002). 
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A  CUSTOM  APPLICATION 

Heuristic:  Tailor  the  application  of  classic  systems  engineering  practices  to  the  specific 
problem  at  hand. 

There  is  not  a  single,  approved  way  to  apply  systems  engineering  to  a  given 
type  of  project.  Each  critical  user  need  or  problem  is  unique.  While  similarities 
may  exist  across  any  set  of  problems,  each  exists  in  a  slightly  different  context 
and  has  its  own  set  of  challenges.  Therefore,  it  is  incumbent  upon  the  systems 
engineers  to  examine  these  discriminating  factors  and  apply  systems  engineering 
accordingly  to  arrive  at  a  suitable  approach.  In  particular,  the  systems  engineer: 
must  understand  the  larger  context  within  which  the  current  project  resides; 
should  look  for  similarities  in  and  borrow  from  other  projects  and  disciplines; 
and  should  select  the  appropriate  tools  for  the  job. 


Keeping  the  feedback  loop  open  and  rapid  proved 

KEY  TO  THE  DECISION  PROCESS. 
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Despite  the  fact  that  each  project  is  unique,  lessons  learned  on  similar 
projects  and  in  other  disciplines  may  prove  useful.  The  FMD  project  looked  to 
the  software  engineering  discipline  for  lessons  learned  and  for  techniques  to 
employ  in  developing  prototypes  where  time  is  short  and  requirements  are  not 
fully  known  or  understood.  Keeping  the  feedback  loop  open  and  rapid  proved 
key  to  the  decision  process. 

Having  the  right  tool  for  the  job  often  makes  a  world  of  difference  in  the 
effectiveness  of  the  effort.  The  FMD  project  needed  a  decision  tool  that  could 
take  the  rapid  feedback  and  continually  provide  an  up-to-date  snapshot  of  the 
solution  space.  The  objectives  hierarchy  was  well  suited  to  this  task.  As  test 
results  came  in  and  were  entered  into  the  tool,  a  new  snapshot  of  the  solution 
space  allowed  the  team  to  continue  to  pursue  promising  technologies  and  drop 
the  ones  that  did  not  perform  well. 


THE  TEAM  INTEGRATOR 

Heuristic:  Systems  engineers  can  integrate  the  team  by  being  the  hub  of  a  collaborative 
process. 

When  a  need  is  critical  and  time  does  not  permit  the  formation  of  a  formal 
team,  groups  may  come  together  in  an  ad  hoc  fashion  to  respond.  The  systems 
engineers  can  help  to  integrate  the  team’s  efforts  by  creating  a  collaborative 
process  and  serving  as  the  hub.  This  role  may  include  responsibilities  such  as 
creating  or  setting  up  collaboration  tools  and  serving  as  the  repository  for 
information.  In  short,  the  systems  engineer  must  treat  the  team  much  like  a 
system  of  systems  that  can  be  integrated  into  a  cohesive  whole. 
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A  USEFUL  RESULT 

Heuristic:  Manage  risk  aggressively,  but  if  no  solution  emerges ,  ensure  that  something 
beneficial  comes  from  the  effort— failure  is  not  an  option. 

Clearly  a  team  would  prefer  to  see  a  viable  solution  emerge  from  the  rapid 
prototyping  process.  Managing  the  risks  in  the  process  is  critical,  just  as  it  is 
in  nearly  any  endeavor.  However,  the  effort  should  not  be  considered  a  failure 
if  a  solution  does  not  emerge.  In  exploring  the  solution  space,  considerable 
knowledge  has  been  gained  and  requirements  are  better  understood.  All 
of  this  knowledge  can  be  fed  into  future  efforts,  allowing  them  to  benefit 
from  that  which  has  gone  before.  Therefore,  the  systems  engineers  must 
aggressively  manage  the  risks  to  increase  the  probability  that  a  solution  will 
be  found,  but  must  also  extract  the  key  lessons  and  knowledge  and  feed  them 
into  future  efforts. 


Managing  risk  requires  knowing  the  “box”  in 

WHICH  THE  PROJECT  MUST  OPERATE. 


Managing  risk  requires  knowing  the  “box”  in  which  the  project  must  operate. 
That  is,  the  team  must  understand  the  key  constraints.  In  so  constraining 
the  effort,  the  team  must  determine  what  must  be  given  up  to  remain  within 
the  box.  On  the  FMD  project,  not  modifying  aircraft  eliminated  a  significant 
portion  of  the  solution  space— the  price  for  meeting  the  schedule  and  budget. 
Understanding  this  box  helped  frame  each  decision. 


Conclusions 

At  the  beginning  of  the  article,  the  question  was  posed:  Can  a  development 
effort  be  responsive  enough  to  react  to  critical  needs  while  still  benefiting  from 
the  rigor  of  systems  engineering?  Experience  from  the  FMD  project  has  shown 
that  an  effort  can  indeed  maintain  the  rigor  of  systems  engineering,  yet  still  be 
nimble  enough  to  react  to  critical  user  needs  in  a  dynamic  environment.  While 
the  approach  taken  for  the  present  effort  will  certainly  not  work  for  every  rapid 
prototyping  effort,  the  key  observations  provide  some  overarching  lessons  to 
guide  future  efforts.  The  heuristics  provided  are  intended  to  be  a  few  more 
tools  in  the  systems  engineering  toolbox  to  aid  the  practitioner  in  applying 
systems  engineering  to  meet  emerging  critical  operational  needs  in  a  rapid 
prototyping  effort. 
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This  article  provides  a  brief  overview  of  Foreign  Military 
Sales  (FMS),  its  role  in  the  ever  changing  dynamic  environ¬ 
ment  that  we  live  in,  and  finally  how  some  of  the  specific 
FMS  processes  can  be  improved  through  the  application  of 
logistics  best  practices  and  initiatives.  The  ultimate  goal  is 
to  continually  improve  the  processes  associated  with  FMS  to 
create  a  win-win  scenario  for  the  Department  of  Defense  and 
affected  stakeholders.  The  best  practices  within  the  frame¬ 
work  of  the  ten  integrated  logistics  support  (US)  elements 
discussed  within  this  article  should  be  merged  into  the 
current  acquisition  or  logistics  support  strategy. 
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Most  civilian  and  military  Department  of  Defense  (DoD)  employees  have 
heard  the  term  "FMS”  sometime  during  their  career.  At  first  glance,  the  concept 
of  Foreign  Military  Sales  (FMS)  appears  simple  and  straightforward— sell 
military  hardware,  software,  services,  or  training  to  a  friendly  nation.  FMS  can  be 
a  valuable  tool  to  supplement  military  cooperation  efforts  and  improve  security 
cooperation  with  friendly  nations.  But  most  notably,  FMS  is  most  often  seen 
merely  as  the  way  we  sell  military  material  to  other  countries.  This  perception, 
however,  is  misleading— there  is  a  lot  more  to  FMS  than  meets  the  eye.  It  is  a 
complicated  process  managed  by  numerous  agencies,  laws,  and  regulations. 
FMS  falls  under  the  umbrella  of  United  States  Security  Assistance,  authorized 
by  the  Foreign  Assistance  Act  (FAA)  of  1961,  and  the  Arms  Export  Control  Act 
(AECA)  of  1976.  The  receiving  country  provides  reimbursement  for  defense 
material  and  services  transferred  from  the  United  States. 


FMS  Management  History 

The  FMS  program  is  that  part  of  Security  Assistance  authorized  by  the 
AECA  and  conducted  using  formal  contracts  or  agreements  between  the  U.S. 
Government  and  an  authorized  foreign  purchaser  (DoD,  2003).  The  Defense 
Reform  Initiative  (DRI)  of  1997  first  coined  the  term  Security  Cooperation.  Since 
the  introduction  of  DRI,  the  overarching  management  responsibilities  for  many 
of  the  DoD-authorized  international  programs  have  for  the  most  part  been 
centralized  and  transferred  to  the  Defense  Security  Cooperation  Agency  (DSCA). 
The  DSCA  was  formerly  known  as  Defense  Security  Assistance  Agency  (DSAA), 
which  primarily  was  responsible  for  many  of  the  Department  of  State’s  Security 
Assistance  programs  authorized  by  the  FAA  and  the  AECA  (DISAM,  2007).  The 
DRI  centralized  various  aspects  of  foreign  Security  Assistance  and  delineated 
key  responsibilities  between  the  State  Department  and  the  DoD.  The  activity  of 
selling  weapon  systems  to  friendly  foreign  governments  becomes  a  leveraging 
tool  of  U.S.  foreign  policy  and  provides  the  United  States  with  an  avenue  to 
conduct  joint  operations  with  the  receiving  nation  (House,  2000).  Executive 
Order  11958  (1977)  allocates  authority  and  responsibility  for  Security  Assistance 
principally  to  the  Secretary  of  Defense  and  the  Secretary  of  State.  The  Secretary 
of  Defense  authority  is  further  delegated  to  the  Under  Secretary  of  Defense 
for  Policy  and  to  the  Director,  DSCA,  in  DoD  Directive  5105.65  (DoD,  2000). 
The  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics 
(USD[AT&L])  supports  and  consults  only  with  DSCA,  and  USD(AT&L)  ensures 
its  strategic  goals  complement  the  two  organizations’  Security  Assistance 
objectives.  Bottom  line:  FMS  is  a  crucial  tool  in  promoting  U.S.  foreign  policy 
and  national  security  interests. 

During  the  cold  war,  U.S.  Security  Assistance  programs  revolved  around 
the  need  to  contain  the  Soviet  Union.  To  this  end,  American  Security  Assistance 
programs  provided  military  training  and  other  support  to  countries  that  U.S. 
policy  makers  viewed  as  essential  to  success  in  the  fight  against  Communism. 
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This  Security  Assistance  approach  changed  after  the  fall  of  the  Berlin  Wall  in 
1989.  The  end  of  the  cold  war  brought  with  it  a  phased  downsizing  of  the  U.S. 
military  starting  in  the  early  1990s.  As  a  result,  the  numbers  of  FMS  agreements 
have  steadily  increased  in  the  last  decade.  The  total  FMS  annual  funding 
increased  from  $8  billion  in  1997  to  $21  billion  in  2006  (DSCA,  2006,  pp.  1-17). 
As  we  can  see  from  this  statistic,  the  growth  of  FMS  would  also  create  a  growth 
in  the  potential  impact  of  best  practices  and  initiatives— specifically,  as  noted  in 
this  article,  to  the  logistics  support  elements. 


FIGURE  1.  FOREIGN  MILITARY  SALES  AGREEMENTS  INCLUDES 
CONSTRUCTION  (DOLLARS  IN  BILLIONS) 

25 


0 

1997  1998  1999  2000  2001  2002  2003  2004  2005  2006 

Year  articles  or  services  are  sold. 

Note:  Information  provided  from  the  Historical  Facts  Handbook  (Defense 
Security  Cooperation  Agency,  2006) 


Figure  1  provides  not  only  an  interesting  trend  in  FMS  growth,  but  an 
increased  opportunity  to  realize  savings  and  efficiencies  “if”  organic  and 
industry  best  practices  are  applied.  Although  we  only  address  three  of  the  ten 
logistical  elements,  the  opportunities  and  impacts  discussed  in  these  three 
elements  indicate  the  potential  for  additional  savings  when  applicable  best 
practices  are  applied  over  the  remaining  seven  elements. 

As  seen  in  Figure  1,  and  based  on  research  conducted  by  the  authors,  it 
was  found  that  the  enhancement  of  logistics  has  significantly  enhanced  the 
effectiveness  of  the  DoD  operations,  as  well  as  those  FMS  cases  that  have  chosen 
to  incorporate  the  enhanced  method.  Where  are  the  results  you  may  ask?  Figure  1 
also  reflects  real  savings  to  be  gained  by  using  economies  of  scale  for  order 
quantities,  incorporating  either  organic  or  industry  capabilities  for  maintenance, 
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and  using  Federal  Express,  United  Parcel  Service,  or  DHL  International  as  a 
source  for  parts  delivery.  As  the  Naval  Aviation  Logistics  Process  Improvement 
Team  (LPIT)  found  out  during  an  effort  to  improve  supportability  of  newly 
procured  weapon  systems  in  the  domestic  and  international  markets— both 
commercial  and  military— the  prime  aircraft  manufacturer  began  offering  hybrid 
support  concepts,  which  some  call  Enhanced  Contractor  Initial  Support  (ECIS), 
for  supporting  the  introduction  of  new  systems.  Per  Bernard  (2003),  the  ECIS 
combined  both  the  aspects  of  what  would  be  considered  a  traditional  U.S. 
Government  support  concept  with  the  Original  Equipment  Manufacturer  (OEM) 
maintenance,  supply  support,  in-service  engineering  support,  and  training 
options.  If  we  are  able  to  successfully  create  an  environment  of  support  for  the 
FMS  customer,  this  could  create  a  cyclical  process  where  the  FMS  customer 
returns  to  request  either  an  increase  of  systems  or  other  available  systems.  In 
this  case,  improved  logistics  support  could  improve  FMS  supportability  but  also 
result  in  new  or  follow-on  FMS  agreements. 

Due  to  the  continued  and  expanded  role  of  FMS,  the  Defense  Acquisition 
University  entered  into  an  agreement  with  the  Taiwanese  Ministry  of  National 
Defense  to  provide  several  courses  related  to  life  cycle  logistics  during  2009, 
and  possibly  beyond.  This  type  of  agreement  to  provide  and  receive  training 
will  only  improve  support  to  our  allies  and  provide  economies  of  scale  to  our 
defense  industry,  making  it  more  efficient.  Also,  through  this  type  of  partnership, 
the  Taiwanese  government  became  aware  of  the  DoD’s  effort  to  incorporate 
Performance-Based  Logistics  (PBL)  as  a  preferred  support  strategy,  and  they 
are  also  looking  to  incorporate  it  into  their  weapon  systems  (J.  Cain,  personal 
communication,  May  28,  2009). 


Potential  FMS  Benefits 

There  can  be  substantial  benefits  in  using  FMS  as  part  of  security 
cooperation  applicable  to  DoD  and  the  partnering  country.  An  example  of  the 
potential  benefits  came  to  light  when  the  former  administration  announced  the 
latest  potential  U.S.  Middle  East  FMS  opportunity.  The  administration  offered 
more  than  $60  billion  in  new  weapons  and  military  assistance  to  Egypt,  Israel, 
Saudi  Arabia,  and  other  U.S.  allies  in  the  Middle  East  (Houska,  2007).  These 
potential  agreements  can  offer  potential  benefits  in  a  variety  of  areas  including 
strategic  partnerships,  mutual  good  will,  as  well  as  a  boost  to  the  U.S.  industrial 
and  economic  base. 

Some  of  the  tools  that  might  be  used  to  support  a  potential  FMS  agreement 
include:  subject  matter  expert  exchanges,  conferences,  large  multinational 
exercises,  where  all  of  these  fall  under  the  larger  umbrella  of  “Security 
Assistance”  and  “International  Cooperation”  (Van  Horn,  2007).  As  we  have  now 
shown,  FMS  is  not  simply  selling  hardware  to  a  friendly  nation. 
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The  Need  to  Improve  FMS 

Improving  cooperation  with  friendly  nations  can  be  brought  about  in 
numerous  ways.  One  way  would  be  to  provide  a  more  effective  and  supportable 
FMS  program.  As  previously  discussed,  the  general  approach  of  any  FMS 
program  should  be  to:  1)  provide  a  faster  and  more  efficient  output  performance 
that  meets  or  exceeds  the  requirements  set  forth  by  the  international  customer; 
2)  take  advantage  of  potential  initiatives  and  existing  best  business  practices; 
and  3)  improve  life  cycle  support.  One  approach  might  be  to  incorporate  the  FMS 
case  into  an  existing  or  new  PBL  agreement.  According  to  Weinberger  (2007), 
the  U.S.  military  can  recount  successes  where  a  PBL  is  established  to  support  the 
international  customer’s  platform  using  a  metric  that  produces  an  outcome  of 
most  benefit  to  the  customer.  Several  international  PBLs  exist  today;  two  worth 
mentioning  are  the  F/A-18E/F  Integrated  Readiness  Support  Teaming  (FIRST) 
contract  that  holds  a  single  OEM  integrator  responsible  for  the  reliability  and 
availability  of  numerous  systems;  and  an  In-Service  Support  Contract  (ISSC) 
with  Boeing,  which  brings  together  common  air  vehicle  sustainment  support 
efforts  for  the  Navy  and  seven  FMS  international  customers. 

The  DSCA  has  been  working  to  transform  FMS  since  the  turn  of  the 
millennium.  The  agency  started  off  in  2000  with  ten  initiatives  under  the 
leadership  of  Lt  Gen  Tome  H.  Walters,  Jr.,  USAF,  [then]  director  of  DSCA.  The 
initiatives  included  the  Civilian  Workforce  Initiatives,  Standby  Letter  of  Credit 
in  Lieu  of  Termination  Liability  Prepayments,  Improved  Payment  Schedule 
Methodology,  and  Team  International.  The  main  objective  of  the  initiatives 
was  to  improve  the  FMS  process  for  both  the  international  customers  and  U.S. 
defense  industry  alike.  While  some  of  these  initiatives  have  been  implemented 
and  others  are  ongoing,  there  is  always  room  for  continuous  improvement.  These 
improvements  will  leverage  advanced  informational  technologies  and  enhance 
the  professionalism  of  the  FMS  workforce  (Beauchamp,  2002).  Informational 
technologies  are  the  backbone  of  the  DoD  logistics  systems,  which  aid  in  the 
management  of  programs  and  help  calculate  requirements  through  use  of 
various  database  systems. 

The  need  to  improve  FMS  was  also  evident  from  the  Navy’s  perspective. 
Naval  Air  Systems  Command  has  a  Naval  Aviation  FMS  LPIT  to  integrate  and 
streamline  the  processes  that  logistically  support  Naval  Aviation  FMS  programs. 
The  LPIT  consists  of  the  FMS  Logistics  Steering  Committee,  the  International 
Logistics  Enterprise  Team,  the  FMS  Customer  Advisory  Group  and  Industry 
Advisory  Group,  and  the  Logistics  Support  Office.  The  various  groups  work 
together  at  conferences  and  in  separate  meetings  to  create  and  enhance  the 
Navy’s  FMS  support  (Bernard,  2003).  This  is  a  classic  example  of  supporting  the 
user  through  improved  logistics  support. 
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Improving  FMS  Through  Logistics 


As  revealed  in  the  previous  paragraphs,  initiatives  at  the  macro  level  are 
available  to  improve  FMS.  Also,  mentioned  earlier  was  the  fact  that  Security 
Assistance  and  FMS  constitute  a  complex  beast,  touching  legal  statutes,  funding 
constraints,  and  competing  agendas  from  different  stakeholders,  similar  to 
the  three  DoD  decision  support  system  processes  of  the  Joint  Capabilities 
Integration  and  Development  System  (JCIDS);  Defense  Acquisition  System;  and 
the  Planning,  Programming,  Budgeting,  and  Execution  (PPBE)  process,  which 
make  up  defense  acquisition. 

Per  DoD  51 05. 38- M  (2000),  the  Department  of  Defense  shall  take  reasonable 
steps  to  support  systems  that  have  been  phased  out  of  DoD  inventory  and 
acquired  by  foreign  nations,  including  items  that  were  never  adopted  by 
U.S.  Forces.  Something  that  can  be  addressed  in  a  relatively  tangible  way  is 
improving  the  logistics  support  to  foreign  countries,  and  creating  a  climate 
of  mutual  support  and  cooperation  among  the  U.S.  Government,  the  U.S. 
defense  industry,  and  the  FMS  countries.  Logistics  can  also  be  construed  as  a 
complex  process,  depending  on  how  it  is  defined.  For  purposes  of  this  article, 
we  use  the  Defense  Acquisition  University’s  ten  Integrated  Logistics  Systems 
(ILS)  elements  (Figure  2),  consisting  of  maintenance  planning;  manpower  and 
personnel;  supply  support;  support  equipment;  training  and  support;  technical 
data;  computer  resources  support;  facilities;  packaging,  handling,  storage, 
and  transportation;  and  design  interface.  The  highlighted  Integrated  Logistics 
Support  (ILS)  elements  (Figure  2)  will  be  discussed  to  show  that  advancements 


FIGURE  2.  INTEGRATED  LOGISTICS  SUPPORT  MODEL 
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Note:  From  the  Defense  Acquisition  University,  Fort  Beivoir,  VA  (2000).  intermediate 
Acquisition  Logistics  course  material. 
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in  defense  supply  support;  maintenance  planning;  and  packaging,  handling, 
storage,  and  transportation  processes  can  improve  FMS. 

As  noted  by  senior  leadership  at  the  Office  of  the  Secretary  of  Defense  level, 
post-production  costs  of  operations  and  maintenance  make  up  approximately 
60-70  percent  of  the  life  cycle  costs  of  a  major  system. 


SUPPLY  SUPPORT 

As  former  military  logisticians,  we  would  be  remiss  if  we  did  not  start  by 
mentioning  the  ever-present  supply  and  parts  shortages,  which  fall  under  the 
ILS  element  of  supply  support.  The  list  of  reasons  why  parts  are  not  available  is 
endless.  In  every  Supply  Chain  section  of  the  Defense  Acquisition  University’s 
LOG  236  (Performance  Based  Logistics)  class  (DAU,  2009),  whenever  a  question 
is  asked  of  the  students  as  to  why  a  system  is  not  available  due  to  parts,  no 
shortage  of  explanations  is  forthcoming.  The  litany  of  explanations  includes  poor 
forecasting,  low  inventory  level,  inaccurate  demand  history,  funding  shortfalls, 
long  lead  items,  unforeseen  surge  or  failures,  sudden  increase  in  demand,  high 
cost,  obsolescence,  diminishing  manufacturing  source,  and  material  shortages. 

Forecasting  affects  all  aspects  of  logistics,  whether  it  be  spares  requirements, 
warfighter  needs,  projected  funding,  flying  hours,  technology  improvement, 
rate  at  which  experienced  workers  will  retire,  or  even  the  types  of  conflicts 
that  war  planners  project— all  are  at  best  mostly  educated  guesses.  While  we 
have  gotten  better  at  forecasting  and  developing  models  to  improve  forecasts, 
forecasting  is  still  an  inexact  science.  Factors  such  as  poor  communication  due 
to  a  language  barrier  or  cultural  differences,  even  with  the  friendliest  of  allies, 
can  magnify  the  forecasting  error  exponentially. 

One  of  the  lesser  known  options  to  improve  logistics  support  for  FMS  is  use 
of  the  Defense  Reutilization  and  Marketing  Service  (DRMS).  With  the  advent  of 
the  Internet,  finding  spare  parts  has  become  lightning  fast,  and  many  Web  sites 
are  available.  However,  the  main  advantage  of  DRMS  is  that  it  receives  over 
$18  billion  worth  of  excess  property  each  year  from  all  armed  services,  which 
is  then  sold  in  an  "as-is"  condition  at  5-50  percent  of  original  value.  FMS  is  one 
of  the  programs  qualified  to  receive  DRMS  property,  subject  to  the  rules  and 
regulations  of  the  AECA  and  the  FAA,  and  all  cases  are  congressionally  notified 
(Schillinger,  1999). 

So  we  have  discussed  the  F-18  FIRST  program,  the  success  of  that  particular 
program,  and  identified  the  flexibility,  efficiencies,  and  economies  of  scale  that 
PBL  brings  to  a  program;  however,  program  success  and  a  successful  logistics 
strategy  are  not  a  be-all/end-all  panacea.  Since  the  Department  of  Defense  sets 
the  requirement  to  define  the  supply  support  strategies,  prudence  dictates  that 
the  Program  Management  Office  investigate  any  and  all  organic  industry  and 
non-DoD  industry  solutions.  Other  issues  have  surfaced  that  will  need  to  be 
addressed  with  international  customers,  specifically  the  repair  and  return  of 
their  equipment.  Cases  have  been  reported  wherein  FMS  customers  turned  in  a 
specific  serialized  item,  expecting  that  the  same  exact  repaired  item  would  be 
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returned.  This  strategy  can  not  be  supported  when  the  PBL  metric  requires  a 
quick  replacement  of  the  component. 

Other  successful  cases  include  the  F/A-18  “FMS  Spares  Call”  that  brings 
together  individual  country  procurements  to  receive  the  financial  benefit 
of  larger  “economies  of  scale”  procurements;  and  the  AIM-9X  combined 
procurement  that  saved  the  U.S.  DoD  over  $3  million  in  cost  avoidance.  The 
United  Kingdom  Ministry  of  Defence  (MoD),  through  a  partnership  with 
Raytheon,  uses  the  best  business  practices  of  single  integrator  to  support  their 
F/A-18  aircraft  by  employing  a  strategy  whereby  incentive  mechanisms  drive 
supplier  performance  and  cost  reductions. 


FIGURE  3.  THREE  TYPES  OF  FMS  SPARES  SUPPORT  MODELS 
(AIR  FORCE  SECURITY  ASSISTANCE  CENTER,  2006) 

Follow-on  Spares  Support 


Result  is  requisitions  filled 
less  than  lead  time  away 


The  basis  of  any  U.S.  Government-sponsored  sale  of  defense  articles  or 
services  is  the  letter  of  offer  and  acceptance  (LOA),  an  agreement  between 
the  two  governments  (seller  and  the  purchaser).  The  LOA  is  commonly  referred 
to  as  an  FMS  case.  As  a  point  of  reference,  three  types  of  spares  support  are 
available  to  FMS  countries  (Figure  3).  First  is  the  Defined  Order  case,  which  is 
most  commonly  used  for  sale  of  major  end  items  that  require  security  controls 
throughout  the  sales  process.  This  is  commonly  referred  to  as  a  standard  sales 
or  firm  order  by  the  U.S.  Army  and  U.S.  Air  Force,  respectively.  Another  option 
is  the  Blanket  Order  case,  which  is  an  agreement  between  a  customer  and  the 
U.S.  Government  to  purchase  a  specific  category  of  items  or  services  at  a  set 
dollar  amount  with  no  definitive  listing  of  the  exact  items  or  quantities  desired. 
Customers  may  requisition  against  a  Blanket  Order  case  as  long  as  the  case  has 
funds  available.  The  final  and  best  method  of  obtaining  spares  support  from  the 
United  States  is  the  use  of  the  Cooperative  Logistics  Supply  Support  Agreement 
(CLSSA).  This  arrangement  is  designed  to  provide  responsive  follow-on  spare 
part  support  for  U.S.  military  hardware  owned  by  foreign  countries. 

The  main  advantage  of  CLSSA  for  a  customer  is  that  it  allows  support  for 
the  purchaser  on  an  equal  basis  with  U.S.  units  having  the  same  force  activity 
designator  (FAD),  which  relates  to  the  mission  of  the  activity  and  the  urgency  or 
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need.  The  caveat  to  this  statement  is  that  even  our  allies  usually  receive  a  lower 
FAD.  But  even  with  a  lower  FAD,  using  the  CLSSA  spares  support  approach  far 
outweighs  the  other  two  support  types.  If  the  Defined  Order  case  approach  is 
taken,  then  the  FMS  customer  might  pay  a  higher  price  instead  of  enjoying  cost 
avoidance  due  to  economies  of  scale,  and  may  expect  an  extended  awaiting 
parts  (AWP)  time.  The  Blanket  Order  case  approach  finds  the  FMS  customer 
having  to  provide  a  substantial  amount  of  cash  to  cover  the  cost  of  components 
not  yet  needed.  There  may  be  some  FMS  customers  that  do  not  have  the  ability 
to  front  large  quantities  of  money  to  cover  unknowns,  or  perhaps  this  approach 
may  be  resisted  for  cultural  reasons.  The  Blanket  Order  case  approach  also 
leaves  the  FMS  customer  with  an  extended  AWP  time.  So  the  bottom  line  benefit 
to  the  CLSSA  approach  is  that  the  FMS  customer  receives  support  on  an  equal 
basis  with  U.S.  units  having  the  same  FAD;  thus,  it  shortens  requisition  fill  times 
(Figure  4)  for  items  that  come  from  DoD  stock  (DISAM,  2007). 


FIGURE  4.  REQUISITION  FILL  TIMES  LESS  THAN  180  DAYS 
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AF  Rep  AF  Rep  DLA  DLA 
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Source:  Air  Force  Security  Assistance  Center,  555th  ILS/Supply  Flight,  CLSSA 
Performance  Brief,  August,  2007. 


The  logistical  element  of  supply  support,  which  in  this  instance  includes 
the  art  of  forecasting  and  spares  support  approach  taken,  has  been  shown  to 
be  a  best  practice  that,  if  managed  properly,  can  be  a  significant  readiness  and 
maintainability  enabler  for  the  FMS  country.  With  the  training  that  is  inherently 
tied  to  most  of  the  ILS  elements,  education  and  better  communication  with 
FMS  customers  will  serve  the  best  interests  of  DoD,  its  allies,  and  affected 
stakeholders.  The  obvious  benefit  of  the  FMS  customer  using  CLSSA  is  the 
consolidated  larger  purchase,  resulting  in  lower  per  unit  cost  for  the  customer. 
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MAINTENANCE  PLANNING 

As  acquisition  and  logistics  practitioners,  we  all  understand  that  when 
designing  an  aircraft,  all  the  maintenance  planning  aspects  are  considered  up 
front  and  early  regarding  what  the  maintenance  requirements  will  be  to  support 
the  system.  However,  when  a  legacy  system  is  sold  to  an  FMS  customer,  how  does 
this  apply  after  the  production  effort  is  complete?  So  let’s  take  a  moment  to  look 
at  several  examples  where  maintenance  planning  efficiencies  were  improved. 

Maintenance  planning  was  used  to  improve  FMS  logistics  support  in  Naval 
Air  Systems  Command’s  In-Service  Support  (ISS).  The  F/A-18  community 
wanted  to  ensure  that  post-production  logistics  and  engineering  support  would 
be  available  for  out-of-production  F/A-18  FMS  customers.  ISS  was  established 
as  a  means  to  keep  the  fleet  modern  and  operationally  viable,  while  continuing 
to  develop  ways  to  reduce  maintenance  costs  and  overcome  the  normal 
obsolescence  of  components  and  systems.  The  ISS  program  has  become  the 
standard  method  that  enables  the  F/A-18  FMS  communities  to  provide  FMS 
customers  with  access  to  the  U.S.  Navy  and  the  prime  contractor  for  long-term 
support  to  the  F/A-18  weapon  system  (Chamberlain,  2000).  In  this  case,  it  needs 
to  be  part  of  the  FMS  case  to  capitalize  on  the  efficiencies  that  can  be  captured 
by  this  approach. 

The  AV-8B  Harrier  is  another  area  that  capitalizes  on  commonality.  Both 
the  U.S.  Marine  Corps  and  the  Italian  Navy  operate  the  AV-8B  using  the  same 
maintenance  practices  and  concepts.  The  approach  of  having  the  same  manuals, 
inspection  intervals,  and  maintainer  qualifications  enables  the  FMS  customer  to 
deviate  from  an  individual  maintenance  concept  that  could  increase  support 
costs.  The  Navy  C-40  Clipper,  a  modified  Boeing  737  platform,  is  another 
example  where  best  commercial  practices  used  by  commercial  airlines  were 
adopted  by  the  Navy.  This  means  that  C-40  Clippers  and  their  crews  can  travel 
worldwide,  confident  that  the  aircraft  can  be  serviced  and  maintained. 

What  we’ve  shown  under  maintenance  planning  is  that  despite  the  fact  that 
a  platform  was  not  designed  with  potential  FMS  applications  does  not  mean 
that  efficiencies  cannot  be  gained  by  using  a  variety  of  organic  government, 
industry,  or  Federal  Aviation  Administration  best  practices.  Thus,  in  the  area 
of  maintenance  planning,  providing  the  FMS  customer  with  long-term  support 
from  the  OEM  (possibly  via  the  use  of  a  Sustaining  Engineering  Contract)  could 
produce  the  framework  to  build  a  long-term  sustainment  support  program.  Key 
to  a  successful  FMS  support  program  is  avoiding  the  loss  of  system  assets  due 
to  a  deficiency  of  maintenance  repair  technical  support.  Again,  we  can  see  the 
potential  for  improvements  applicable  to  the  FMS  arena. 


PACKAGING,  HANDLING,  STORAGE,  AND  TRANSPORTATION 

When  talking  about  transportation,  we  first  must  understand  what  is 
available  from  the  air  and  sea  under  the  Defense  Transportation  System  (DTS). 
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The  U.S.  Transportation  Command  (USTRANSCOM)  consists  of  three  elements. 
The  Air  Mobility  Command  transports  material  and  personnel  around  the  world 
through  both  organic  and  commercial  contracted  air  carriers.  The  Military 
Sealift  Command  transports  material  around  the  world  through  organic  and 
contracted  commercial  surface  ships.  The  Surface  Deployment  and  Distribution 
Command  operates  the  military  ports  in  both  the  United  States  and  overseas. 
These  three  elements  are  referred  to  as  the  DTS  (Figure  5). 


FIGURE  5.  USTRANSCOM  AND  THE  DEFENSE  TRANSPORTATION 
SYSTEM 
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USTRANSCOM  is  responsible  for  the  movement  of  about  560  tons  of  freight 
per  day.  However,  actual  FMS  shipments  comprise  only  about  6  percent  of 
USTRANSCOM's  annual  business.  For  a  variety  of  reasons,  in-transit  visibility  is 
not  consistently  available  to  FMS  customers,  as  noted  by  the  Defense  Institute 
of  Security  Assistance  Management  (DISAM,  2007).  The  DoD  prefers  not  to 
be  involved  in  the  movement  of  FMS  material  and  encourages  customers  to 
be  self-sufficient  in  arranging  for  transportation  and  obtaining  in-transit 
visibility  data.  USTRANSCOM  has  come  up  with  a  solution  for  better  visibility 
and  accountability  for  FMS  material  through  the  Enhanced  Freight  Tracking 
System  (EFTS).  Per  DSCA  Memorandum  (2008),  EFTS  is  a  secure  Web-based 
application  that  provides  in-transit  visibility  of  FMS  shipments.  Resident  in  the 
Security  Cooperation  Information  Portal,  EFTS  serves  as  a  consolidated  source 
for  FMS  in-transit  information.  Ultimately,  EFTS  applications  will  evolve  to 
provide  visibility  of  the  FMS  distribution  pipeline  for  all  classes  of  supply  and 
modes  of  transportation,  with  tracking  visibility  of  outbound  cargo  from  the 
United  States  to  the  FMS  customer's  country  or  cargo  returning  to  the  United 
States  or  a  U.S  facility  overseas. 

Now  that  we  have  an  understanding  of  what  is  available  within  the  defense 
and  transportation  sectors,  what  of  the  transportation  experts  that  support 
the  global  economy?  Can  the  advantages  that  they  bring  to  transportation  be 
harnessed  and  capitalized  on  to  benefit  not  only  the  Department  of  Defense 
systems,  but  also  FMS?  We  need  to  understand  the  capabilities  that  FedEx, 
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UPS,  or  DHL  might  bring  to  this  equation.  These  commercial  delivery  services 
may  serve  as  strong  enablers  in  moving  requirements  to  and  from  major  staging 
areas.  But  in  actuality,  the  supporting  program  office  must  understand  the  entire 
transportation  scope  in  order  to  apply  the  commercial/industrial  capabilities  in 
an  effective  manner.  Anyone  now  has  the  capability  to  go  online,  order  an  item, 
and  have  it  shipped  to  their  location  from  practically  anywhere  in  the  world, 
quickly  and  conveniently.  In  many  cases,  tracking  visibility  is  also  offered  online. 
Obviously,  the  potential  of  increased  efficiency  of  transportation  can  be  gained 
by  applying  the  best  aspects  of  the  DTS  and  the  global  transportation  carriers. 


Conclusions 

This  article  provides  a  broad  overview  of  FMS  and  its  importance  to  the 
U.S.  Government,  defense  industry,  and  allied  nations;  and  how  FMS  processes 
can  be  improved,  specifically  through  logistics.  While  only  three  of  the  ten 
ILS  elements  were  addressed— supply  support;  maintenance  planning;  and 
packaging,  handling,  storage,  and  transportation— any  of  the  other  ILS  elements 
can  be  leveraged  to  improve  FMS  support.  This  article  shows  that  FMS  support 
can  be  improved  through  logistics  via  advancements  in  both  organic  and 
commercial  capabilities.  The  authors’  intent,  however,  is  that  it  serve  as  a  catalyst 
for  program  managers,  their  allied  counterparts,  and  affected  stakeholders  to 
pursue  improvement  in  logistics  processes,  thereby  increasing  weapon  systems 
supportability  and  accelerating  support  to  joint  warfighters. 
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A  MULTI-CRITERIA 
DECISION  MODEL  FOR 
MIGRATING  LEGACY 
SYSTEM  ARCHITECTURES 
INTO  OPEN  SYSTEM  AND 
SYSTEM-OF-SYSTEMS 
ARCHITECTURES 


\'  Cyrus  Azani 

Full-spectrum  dominance  in  battlefield,  cost-effective  devel¬ 
opment  of  capabilities,  timely  reaction  to  evolving  threats 
and  technologies,  and  system  and  process  flexibility  can  be 
greatly  enabled  through  the  application  of  open  systems 
strategies.  Such  strategies  are  effective  business  and  technical 
approaches  for  assessing  the  appropriateness  of  developing 
modular  and  open  architectures  for  stand-alone  as  well  as 
system  of  systems.  This  article  will  introduce  an  integrated 
methodology  for  assessing  existing  systems  and  migrating 
their  architectures  into  modular  and  open  architectures.  The 
proposed  method  integrates  open  systems  strategies  with 
the  Analytic  Hierarchy  Process  (AHP)  and  Goal  Programming 
(GP)— two  powerful  decision-making  models— to  evaluate 
the  appropriateness  of  open  systems  migration,  rank  migra¬ 
tion  candidates,  allocate  resources  among  them,  and  develop 
open  architectures  for  selected  systems. 


Keywords:  Legacy  Migration,  Open  Architecture, 
System  of  Systems  Engineering,  Analytic  Hierarchy 
Process  (AHP),  Goal  Programming  (GP) 
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Timely  reaction  to  evolving  threats  and  technology  requires  agile  system 
architectures  that  could  quickly  and  cost-effectively  be  integrated  and 
reconfigured  within  family  of  systems  and  joint  system  of  systems  warfighting 
constructs.  Affordable  agility  and  reconfiguration  will  demand  open  and 
modular  forces,  systems,  and  system  of  systems  (SoS).  The  recent  change  in 
U.S.  Army  strategy  from  division-centric  structures  to  modular  brigade  combat 
teams;  publication  of  DoD  Directive  5000.01  (DoD,  2003),  which  mandated 
implementation  of  the  Modular  Open  Systems  Approach  (MOSA);  the  Naval  OA 
(Open  Architecture)  Strategy  (2008);  and  the  Open  Technology  Development 
Roadmap  Plan  (Department  of  Air  Force,  2006)— all  are  testimony  to  a  shift 
in  warfighting  and  acquisition  within  the  DoD.  If  applied  effectively,  an  open 


This  article  proposes  an  integrative  mode/ 

METHOD  FOR  MIGRATING  LEGACY  SYSTEMS  INTO  OPEN 
SYSTEMS  IN  A  FAMILY  OR  SYSTEM  OF  SYSTEMS  CONTEXT. 


and  modular  architecture  can  enhance  the  adaptability  of  defense  systems  to 
changes  in  threats  and  technology,  reduce  the  total  ownership  costs  of  systems, 
and  improve  life  cycle  supportability.  Moreover,  by  following  an  open  system 
strategy  in  acquisition  of  systems,  the  programs  will  be  in  a  better  position  to 
leverage  investments  made  throughout  the  defense  industrial  base  to  produce 
new  commercial  products,  practices,  and  technologies  that  will  integrate 
warfighting  capabilities  more  easily  in  a  system  of  systems  environment  and 
field  superior  capabilities  more  quickly  and  affordably.  Furthermore,  an  open 
system  strategy  considers  life  cycle  support  requirements  up  front,  permits 
system  evolution  with  technology  development,  opens  the  diminishing  defense 
industrial  base  to  commercial  know-how  and  products,  anticipates  technology 
obsolescence  in  system  design,  and  supports  continuing  technology  insertion 
throughout  the  system  life  cycle. 

The  best  strategy  for  developing  an  open  system  architecture  is  to  follow 
the  MOSA,  originally  proposed  by  Azani  and  Flowers  (2005).  This  approach  is  an 
integrated  business  and  technical  strategy  that  employs  a  modular  design  and, 
where  appropriate,  defines  key  interfaces  using  widely  supported  and  consensus- 
based  standards  that  are  published  and  maintained  by  recognized  industry 
standards  organizations  (i.e.,  open  standards).  By  following  this  approach,  the 
programs  will  first  make  a  business  case  for  open  systems  and  then,  through 
adherence  to  five  major  MOSA  principles,  develop  an  open  architecture  for  the 
system  under  consideration  (Azani  &  Khorramshahgol,  2005). 
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FIGURE  1.  THE  MODULAR  OPEN  SYSTEMS  APPROACH 
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Figure  1  depicts  the  DoD  vision,  the  fundamental  principles,  and  proven 
benefits  of  following  MOSA.  As  shown  in  Figure  1,  MOSA  must  become  an 
integral  part  of  each  acquisition  strategy  to  achieve  affordable,  evolutionary, 
and  joint  combat  capability.  Effective  implementation  of  MOSA  is  dependent 
on  adherence  to  five  fundamental  MOSA  principles  (Azani  &  Flowers,  2005). 
Additional  principles  have  been  added  to  this  list  which  include  biotic  open 
systems  that  have  been  around  for  millions  of  years  (Azani,  2009).  Although 
considerable  literature  exists  on  developing  an  open  architecture  for  a  new 
system  (Azani,  2009;  Azani  &  Flowers,  2005;  Azani,  2000,  2001a,  2001b;  Azani  & 
Khorramshahgol,  2005a,  2005b,  2005c,  2006),  no  coverage  of  legacy  systems 
migration  into  open  systems  is  evident.  Also  lacking  are  proven  models  or 
methodologies  to  evaluate  the  feasibility  of  a  migration  decision,  identification 
of  appropriate  migration  candidates,  and  prioritization  and  implementation  of 
open  architectures  for  the  selected  candidate  systems. 

This  article  proposes  an  integrative  mode/method  for  migrating  legacy 
systems  into  open  systems  in  a  family  or  system  of  systems  context.  The  proposed 
model/method  is  equally  applicable  to  migration  decisions  involving  subsystems 
in  a  single  system.  The  model/method  integrates  the  Modular  Open  Systems 
Approach  (MOSA)  with  two  powerful  mathematical  models,  namely  the  Analytic 
Hierarchy  Process  (AHP)  and  Goal  Programming  (GP).  By  incorporating  the  MOSA 
concept  into  the  model,  decision  makers  will  also  enable  their  programs  to:  1) 
meet  challenges  associated  with  integrating  technologies  from  different  vendors; 
2)  maintain  continued  access  to  cutting-edge  technologies  and  products  from 
multiple  suppliers;  3)  facilitate  quick  development  and  cost-effective  change  of 
legacy  applications;  and  4)  address  change  management  as  system  requirements 
evolve  and  new  technologies  become  available.  Through  incorporating  the  two 
proven  mathematical  models,  the  suggested  method  enables  organizations 
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to  effectively  consider  multiple  criteria  and  conflicting  goals  in  the  decision¬ 
making  process  and  integrate  tangible  as  well  as  intangible  factors  from  various 
stakeholders  to  reach  a  more  acceptable  and  best  possible  decision. 

The  following  sections  present  a  brief  introduction  to  an  open  system 
concept  and  discuss  the  legacy  migration  challenges  organizations  face.  The 
AHP  and  GP  are  then  explained.  Finally,  the  proposed  method  is  discussed  and 
its  application  illustrated  through  an  example,  followed  by  a  discussion  of  the 
advantages  and  limitations  of  the  proposed  methodology. 


What  is  an  Open  System  Concept? 

The  open  system  concept  evolved  from  biological  and  physical  sciences 
and  was  adopted  in  the  information  technology  industry  in  the  late  1960s  and 
early  1970s  (Azani,  2000).  For  many  years,  information  systems  buyers  were 
limited  to  only  a  few  major  mainframe  vendors,  of  which  one  vendor  was  clearly 
dominant  in  the  marketplace.  Competition  was  severely  limited  because  access 
to  the  market  was  controlled  essentially  by  one,  or  no  more  than  a  few,  vendors 
(Azani  2002).  A  number  of  different  standards  organizations  initiated  open 
systems  efforts,  sometimes  in  competition  with  one  other.  Recently,  some  order 
has  come  to  the  scene,  and  some  degree  of  convergence  appears  reasonable. 

Open  system  definitions  vary  within  disciplines,  industries,  and 
organizations.  Nevertheless,  there  are  some  common  themes  that  most  of  these 
definitions  contain  (Roark  &  Kiczuk,  1995).  Generally  speaking,  open  systems 
are  systems  with  permeable  boundaries  or  well-defined  standardized  interfaces 
that  enable  exchange  of  energy  (e.g.,  electrical  current  via  a  wall  plug);  material 
(e.g.,  replacement  of  components/parts  with  equivalent  components  from 
competitive  sources);  and  information  (e.g.,  interoperability  with  other  systems) 
within  the  joint  operating  environment  (Azani,  2009).  Within  this  context, 
open  systems  are  defined  as  systems  that  employ  modular  design,  use  widely 
supported  and  consensus-based  standards  for  their  key  interfaces,  and  are 
subjected  to  successful  validation  and  verification  tests  to  ensure  the  openness 
of  their  key  interfaces  (Azani  &  Flowers,  2005).  Open  architectures  enable  easier 
integration  of  properly  engineered  modules  across  a  wide  range  of  systems, 
effective  reconfiguration  and  reintegration  into  joint  warfighting  and  system 
of  systems  constructs,  successful  exchange  of  information  and  services  with 
other  modules  on  local  and  remote  systems,  and  more  affordable  and  quicker 
adaptation  to  evolving  needs  and  technologies. 


What  is  an  Open  Architecture? 

Architecture  means  different  things  to  different  people.  Some  definitions 
are  complex  and  depicted  in  very  complicated,  confusing,  and  long  sentences 
(Jazayeri  &  Linden,  2000;  Booch  et  al.,  1999).  Other  definitions  such  as  the  ones 
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proposed  by  Rechtin  and  Maier  (1997)  are  simpler  and  more  concise  definitions 
of  architecture.  The  best  architecture  definition  is  perhaps  the  definition 
proposed  by  recognized  standards  organizations  such  as  the  one  by  the 
Institute  of  Electrical  and  Electronics  Engineers  that  defines  architecture  as  the 
structure  of  components,  their  relationships,  and  the  principles  and  guidelines 
governing  their  design  and  evolution  over  time.  However,  the  architecture  of 
a  given  system  seems  to  be  not  only  the  structure  of  the  system,  but  also  its 
functions,  the  environment  in  which  it  will  reside,  and  the  process  by  which  it 
will  be  built  and  operated  (Rechtin,  1997).  It  also  represents  the  highest-level 
conception  of  a  system  in  its  environment,  includes  the  structure  and  behavior 
of  the  whole  system  in  that  environment,  and  how  it  will  meet  its  requirements 
(Emery  et  a  I 1996). 

An  open  architecture  depicts  the  structure  of  functional  and  physical 
modules,  their  interrelationships  through  well-defined  open  key  interfaces, 
and  the  principles  governing  the  design,  development,  reconfiguration,  and 
evolution  of  such  structure  over  time.  Open  architectures  rely  on  physical 
modularity  and  functional  partitioning  of  both  hardware  and  software  to 
create  the  flexibility  needed  for  replacing  specific  subsystems  and  components 
without  affecting  others.  The  open  architecture  supports  the  functional 
baseline  and  system  specifications  and  is  an  effective  blueprint  for  developing 
and  maintaining  affordable  and  adaptable  applications  and  systems.  Moreover, 
some  organizations  are  at  a  higher  level  of  maturity  in  their  application  of 
open  architecture,  while  others  operate  at  very  early  levels  or  stages  of  open 
systems  maturity  (Azani,  2002).  By  developing  open  architectures,  the  system 
integrators/architects  will  build  flexibility  into  their  systems/applications  and 
will  achieve  enduring  interoperability,  integrability,  affordability,  adaptability, 
and  supportability  for  their  systems. 


Migration  Issues  and  Challenges 

Organizations  face  a  number  of  formidable  challenges  in  their  decision 
making  process  of  migrating  legacies  into  open  systems: 

•  How  does  an  organization  determine  which  legacy  systems 
should  be  kept,  modernized,  and  be  migrated  into  open  systems? 

•  How  does  the  organization  tackle  the  integration  of  legacy 
systems  into  joint  warfighting  system  of  systems  constructs? 

•  How  does  the  organization  mitigate  the  risks  associated  with 
obsolescence  and  unavailability  of  components  comprising  a 
legacy  system? 

•  How  does  the  organization  avoid  the  legacy  dependence  on  a 
single  source  of  supply  for  the  remainder  of  legacy  useful  life? 

•  In  case  the  organization  decides  to  keep  the  functionality  of 
legacy  systems,  how  does  the  organization  reach  consensus 
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on  objectives  and  criteria  needed  for  analyzing  the  existing 
functionality,  evaluating  their  future  relevance,  and  assessing  the 
feasibility  of  their  migration  into  open  systems? 

•  How  should  the  organization  prioritize  migration  candidates, 
allocate  resources,  and  develop  open  architectures  for  them? 

•  How  does  the  organization  deal  with  complexity  and  risks  of 
migration  and  integration  considering  the  many  applications, 
diverse  networks,  various  standards,  and  numerous  platforms 
that  exist  in  a  typical  organization? 

•  How  does  the  organization  deal  with  multiple  conflicting 
objectives  in  migrating  legacies  into  open  systems? 

Other  challenges  that  decision  makers  face  are  to  establish  objective  criteria 
to  compare  different  legacy  systems  and  select  the  migration  candidates. 
Examples  of  such  criteria  are  remaining  useful  life  of  the  legacy  system,  expected 
life  cycle  cost  savings  as  a  legacy  system  is  migrated  into  an  open  system,  and 
the  extent  of  future  risk  avoidance.  Unfortunately,  in  most  organizations  the 
criteria  used  for  comparison  are  subjective,  and  decision  makers  cannot  reach 
a  consensus  on  the  magnitude  of  risks,  costs,  or  system  useful  life.  They  are 
also  less  likely  to  agree  on  likelihood  of  occurrence  of  various  types  of  risks 
encountered  if  the  legacy  system  is  not  migrated. 

Seamless  integration  of  legacy  systems  can  be  a  very  complex  and  tedious 
project,  across  numerous  applications,  diverse  networks,  various  standards,  and 
many  platforms  in  an  organization.  System  integration  not  only  is  a  challenging 
task,  it  is  also  quite  risky.  Not  surprisingly,  88  percent  of  integration  projects 
fail  (Pollock,  2001).  Most  if  not  all  of  the  legacy  system  integration  challenges 
mentioned  above  can  effectively  be  addressed  by  using  the  proposed 
methodology  and  by  making  every  system  a  part  of  an  integrated  network 
of  open  architectures.  The  proposed  methodology  will  rely  on  an  integrated 
product  and  process  teaming  arrangement  to  establish  agreed-upon  objectives 
and  criteria  needed  for  comparison  of  legacy  systems.  After  establishing 
objectives  and  criteria,  the  problem  would  be  to  prioritize  migration  candidates 
based  on  these  criteria  and  allocate  resources  among  them.  To  this  end,  the 
proposed  methodology  applies  AHP  for  prioritization  of  legacy  systems  and 
GP  for  resource  allocation.  A  powerful  feature  of  GP  is  its  ability  to  allow  for 
consideration  of  multiple  conflicting  goals  and  objectives  when  allocating 
organizational  resources. 


Analytic  Hierarchy  Process  (AHP) 

Analytic  Hierarchy  Process  (Saaty  1980)  is  based  on  the  idea  that  a  complex 
issue  can  be  effectively  examined  if  it  is  hierarchically  decomposed  into  its 
parts.  AHP  implementation  entails  a  hierarchy  whose  top  level  reflects  the 
overall  objective.  Criteria  are  listed  at  intermediate  levels,  while  the  lowest  level 
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represents  the  alternatives.  Elements  at  a  certain  level  are  compared  to  one 
another  with  reference  to  their  effect  on  the  higher  level.  Saaty  (1994, 1996)  has 
an  inconsistency  index  to  capture  any  bias  when  relative  comparisons  are  made. 
A  zero  value  would  indicate  perfect  consistency,  whereas  larger  values  indicate 
increasing  levels  of  inconsistency.  Saaty  (1994)  states  that  the  Inconsistency 
Ratio  (IR)  should  be  about  10  percent  or  less  to  be  acceptable.  If  the  IR  exceeds 
the  10  percent  level,  value  judgment  may  need  to  be  revised.  AHP  has  been 
applied  in  a  number  of  areas  ranging  from  engineering,  economics,  and  politics, 
to  marketing,  sociology,  and  management.  Vargas  and  Zahedi  (1993)  present  a 
comprehensive  survey  of  AHP  and  its  applications. 

AHP  is  a  valuable  component  of  the  proposed  methodology  because:  a)  it 
considers  the  legacy  system  migration  problem  in  its  entirety;  b)  it  breaks  the 
problem  down  into  its  components  and  subcomponents;  and  c)  it  incorporates 
multiple  (conflicting)  objectives,  uncertainty,  and  decision  makers’  preferences 
in  the  decision-making  process.  In  addition,  AHP  allows  for  multiple  stakeholders 
to  participate  in  the  decision  making  process.  For  excellent  discussions  of  AHP 
applications,  see  Saaty  (1994,  1996),  Liberatore  (1987),  Vargas  and  Zahedi 
(1993),  Liberatore  and  Nydick  (2003),  and  Wasil  and  Golden  (1991). 


Goal  Programming 

GP  deals  with  allocating  scare  resources  among  alternatives  in  the  best 
possible  way,  by  mathematically  stating  the  problem  so  as  to  minimize  a  given 
function  subject  to  a  set  of  constraints.  The  procedure  starts  with  specifying  a 
target  or  aspiration  value  for  each  objective,  thus  transforming  all  objectives 
into  goals.  The  resultant  objective  function— termed  achievement  function— is 
then  the  summation  of  deviations  from  these  goals.  Since  attaining  all  goals 
simultaneously  is  impossible,  the  problem  is  then  to  minimize  the  sum  of  the 
deviations  from  the  goals;  that  is,  to  minimize  the  achievement  function. 

GP  initially  was  developed  by  Charnes  and  Stedry  (1964)  and  extended 
and  improved  by  Jaaskelainen  (1969)  and  Ignizio  (1976,  1982).  Ijiri  (1965)  also 
suggested  the  concept  of  “preemptive  priorities,”  where  a  priority  is  given  to  an 
objective  or  a  set  of  commensurable  objectives. 

The  GP  model,  as  described  by  Ignizio  (1982),  follows: 

Find  x  =  x1, . ,Xj, . Xj  so  as  to  minimize: 

a  =  g,(n,p), . gk(n,p), . gK(n,p) 

such  that: 

f(x)  +  a  -  pi  =  b.  for  all  i  =  1, . ,m 

and  x,  n,  p  >0, 

where: 

“xj”  is  the  jth  decision  variable, 
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“a”  is  denoted  as  the  achievement  function;  a  row  vector  measure  of 
the  attainment  of  the  goals  or  (rigid)  constraints  at  each  priority  level, 
“gK(n,p)”  is  a  function  (normally  linear)  of  the  deviation  variables 
associated  with  the  objectives  or  constraints  at  priority  level  k, 

“K”  is  the  total  number  of  priority  levels  in  the  model, 

“b”  is  the  right-hand  side  constant  for  goal  (or  rigid  constraint)  i, 

“f(x)”  is  the  left-hand  side  of  the  linear  or  nonlinear  goal  or  constraint  i. 


The  Proposed  Methodology 


As  mentioned  earlier,  the  proposed  methodology  employs  the  MOSA 
concept,  applies  an  Integrated  Product  and  Process  Team  (IPPT)  approach,  and 
uses  AHP  and  GP  models.  Such  concepts  and  models  are  integrated  together 
to  evaluate,  plan,  and  monitor  the  application  of  the  methodology;  set  priorities 
among  legacy  candidates;  allocate  resources  among  them;  and  finally,  develop 
open  architectures  for  the  selected  migration  candidates. 

The  major  constructs  of  the  proposed  method  are  depicted  in  Figure  2  and 
its  practicability  and  applicability  will  be  demonstrated  by  a  hypothetical  naval 
organization  in  subsequent  section  of  this  article. 

The  methodology  utilizes  three  major  phases  and  a  number  of  steps  within 
each  phase,  as  detailed  in  the  following  discussion. 

FIGURE  2.  THE  PROPOSED  MIGRATION  METHODOLOGY 
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PHASE  1:  MAKE  A  BUSINESS  CASE  FOR  MIGRATION  AND  ESTABLISH  A 
MIGRATION  PLAN 

At  this  phase,  the  team  that  oversees  the  application  of  the  methodology 
is  appointed,  legacy  system  candidates  are  identified  and  prioritized,  the  GP 
problem  is  formulated,  and  organizational  resources  will  be  allocated  among 
selected  migration  candidates. 

Step  1.1.  Use  an  IPPT  to  identify  legacy  systems  in  need  of  migration  and  assess 
their  functionality;  determine  which  legacy  system  should  be  kept,  modernized, 
or  eliminated;  establish  migration  objectives  and  evaluation  criteria;  and  oversee 
the  entire  migration  process.  The  IPPT  may  select  diverse  and  conflicting 
objectives.  However,  this  will  not  present  any  problem  as  GP  allows  for  multiple, 
conflicting,  and  non-commensurable  objectives  of  the  organization  to  be 
included  in  problem  formulation.  The  preferred  IPPT  format  will  be  a  Delphi 
inquiry  to  remove  Groupthink  and  bias  from  the  process,  assure  anonymity  and 
continuing  feedback,  and  allow  for  a  more  refined  and  comprehensive  analysis 
of  the  problem  (Linstone  &  Turoff,  1975). 

Step  1.2.  Apply  AHP  to  set  priorities  among  the  legacy  systems  identified  earlier 
using  the  evaluation  criteria  identified  in  Step  1.1.  In  other  words,  the  set  of 
legacy  system  candidates  generated  in  the  previous  step  is  presented  to  the 
members  of  the  IPPT  to  obtain  their  subjective  value  judgments  for  a  pair¬ 
wise  comparison  matrix.  The  eigenvalues  of  this  matrix  represent  the  priorities 
among  the  criteria  selected  as  well  as  the  priorities  among  the  various  legacy 
system  candidates.  The  priorities  among  the  selected  criteria  will  then  be  used 
as  penalty  weights  in  the  objective  function  of  the  GP  model. 

Step  1.3.  Using  the  information  obtained  in  the  two  previous  steps,  the  IPPT 
will  formulate  a  GP  model  to  allocate  resources  among  the  selected  migration 
candidates. 

Step  1.4.  Establish  proper  migration  plans  and  strategies  (acquisition,  logistics, 
test  and  evaluation,  etc.)  to  cost  effectively  transform  the  selected  legacies  into 
open  systems.  The  migration  strategies  should  also  specify  when,  how,  and  in 
what  order  the  migration  efforts  should  proceed. 


PHASE  2:  DEVELOP  OPEN  ARCHITECTURES  FOR  MIGRATING  SYSTEMS 

At  this  phase,  through  compliance  with  the  five  MOSA  principles 
identified  earlier,  an  open  architecture  will  be  developed  for  each  approved 
legacy  system  candidate. 

Step  2.1.  Establish  an  enabling  environment.  Effective  MOSA  implementation 
is  contingent  upon  adequate  skills  and  training  on  the  open  systems  concept; 
suitable  acquisition  and  logistics  verification  and  validation  strategies;  and 
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establishment  of  an  appropriate  MOSA  implementation  roadmap  with  milestones 
and  performance  measures. 

Step  2.2.  Employ  modular  design  tenets  to  repartition  the  legacy  system  into 
encapsulated,  cohesive,  and  self-contained  modules  with  well-defined  internal 
and  external  interfaces. 

Step  2.3.  Group  interfaces  into  key  and  non-key  interfaces  using  criteria  such  as 
the  rapid  rate  of  technology  turnover,  high  cost,  interoperability  requirement, 
and  the  failure  rate  of  modules  at  each  end  of  an  interface. 

Step  2.4.  Use  widely  supported  and  consensus-based  (open)  standards  for  key 
system  interfaces  to  develop  an  open  architecture  for  the  system. 

Step  2.5.  Use  proper  verification  and  validation  mechanisms  to  ensure  openness 
of  key  interfaces  and  the  overall  system  architecture. 


PHASE  3:  GAUGE  THE  PROCESS  AND  TAKE  CORRECTIVE  ACTION 

During  this  phase  proper  contracting  language,  performance  measures, 
and  validation  criteria  are  established  to  ensure  openness  of  the  selected 
migration  systems. 

Step  3.1  Use  appropriate  contracting  language  (e.g.,  section  L  and  section  M 
stipulations)  to  ensure  that  subsystems,  components,  and  commercial  products 
delivered  are  open. 

Step  3.2.  Use  appropriate  performance  measures  (metrics)  to  assess  the  MOSA 
implementation  progress  and  system  openness.  Examples  of  such  metrics  are 
the  percentage  of  key  interfaces  defined  by  open  standards,  and  percentage  of 
system  modules  that  can  be  acquired  from  multiple  competitive  sources  when 
the  system  is  migrated. 

Step  3.3.  Validate  that  the  migration  goals  and  objectives  are  achieved. 


Application  of  the  Proposed  Model 

To  demonstrate  its  practicability  and  applicability,  the  methodology  is 
applied  to  a  hypothetical  SoS  portfolio  of  existing  naval  systems.  Let  us  assume 
that  the  IPPT  selected  by  the  SoS  Program  Executive  Office  has  decided  to 
use  the  proposed  model/method  and  has  conducted  an  inquiry  and  reached  a 
consensus  on  the  following  organizational  objectives/criteria  to  be  pursued  for 
the  next  ten  years: 
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Goal  1.  Reduce  total  ownership  cost— measured  by  the  net  present  worth  of 
total  projected  cost  saving/avoidance  resulting  from  gaining  access  to  multiple 
sources  of  supply  and  migration  to  an  open  system  architecture. 

Goal 2.  Reduce  obsolescence  risks— measured  by  the  number  of  open  standard- 
compliant  products  in  the  migrated  system. 

Goal  3.  Improve  availability  and  reliability— measured  by  an  increase  in  system 
reliability  and  availability  resulting  from  the  latest  COTS  hardware  and  software 
products  enabled  and  employed  by  the  migrated  system. 

Goal  4.  Improve  system  capability— measured  by  the  percentage  improvement 
in  the  overall  system  capability  when  its  architecture  is  migrated  into  an  open 
systems  architecture.  Figure  3  shows  a  recommended  approach  for  measuring 
risks  and  the  impacts  on  capability  if  a  legacy  system  does  not  become  open. 


FIGURE  3.  THE  AHP  MODEL  FOR  THE  ILLUSTRATIVE  EXAMPLE 

O  Select  Candidates  for  OS  Migration 


Cost 

1  Obsolescence  Risk 
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Capability 

Improved  SoS 
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Goal  5.  Enhanced  integration  and  interoperability— measured  by  the  number  of 
key  internal  and  external  interfaces  that  will  be  defined  by  open  standards  as 
the  system  migrates  into  an  open  architecture. 

Let  us  assume  that  the  funds  allocated  among  the  candidate  legacies  must 
be  proportional  to  the  level  of  contribution  of  each  legacy  system  toward  the 
achievement  of  the  goals  listed  above.  This  requirement  will  bring  four  new 
constraints  into  the  GP  formulation.  Table  1  depicts  the  target  level  for  each 
objective/goal  as  specified  by  the  IPPT. 

Let  us  further  assume  that  goals  1,  2,  and  4  must  be  achieved,  at  a  minimum, 
by  the  amount  specified,  and  an  exact  achievement  of  goals  3  and  5  is  desired. 

Let  us  assume  that  after  careful  consideration  of  the  functionality, 
modernization  options,  and  life  expectancy  of  all  the  legacy  systems  in  the  SoS 
portfolio  of  naval  systems,  four  hypothetical  legacy  systems  (i.e.,  LHA  10,  LPD  12, 
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TABLE  1.  THRESHOLDS/TARGET  LEVELS  FOR  EACH  GOAL 


Objective 

Target  Level 

Improve  Cost  Effectiveness  (at  least) 

$12,000,000 

Reduction  in  Obsolescence  Risk  (at  least) 

100 

Improve  Overall  User  Satisfaction 

20% 

Improve  Overall  Systems  Capability  (at  least) 

60% 

Enhance  System  of  Systems  (SoS)  Integration 

80 

Use  AHP  to  assign  weights  to  these  goals. 

Use  Goal  Programming  to  minimize  deviation  from  these  goals  and  compute  fund 
allocation  percentages. 


LCC  50,  and  LSD  70)  were  found  to  be  suitable  candidates  for  migration  to  open 
systems.  Figure  4  depicts  the  hierarchy  structure  of  the  problem. 

Let  us  further  assume  that  the  following  preferences,  based  on  Table  2, 
were  elicited  from  the  participants  of  the  open  systems  migration  IPPT  or  Delphi 
inquiry  using  the  AHP  preference  criteria: 

•  Criterion  1  is  very  strongly  preferred  to  criterion  2 

•  Criterion  1  is  strongly  preferred  to  criterion  3 

•  Criterion  1  is  equally  to  moderately  preferred  to  criterion  4 

•  Criterion  1  is  extremely  preferred  to  criterion  5 

•  Criterion  2  is  moderately  preferred  to  criterion  5 

•  Criterion  3  is  strongly  preferred  to  criterion  2 

•  Criterion  3  is  moderately  to  strongly  preferred  to  criterion  5 

•  Criterion  4  is  very  strongly  preferred  to  criterion  2 

•  Criterion  4  is  strongly  preferred  to  criterion  3 

•  Criterion  4  is  extremely  preferred  to  criterion  2 


TABLE  2.  PAIRWISECOMPARISON  MATRIX  FOR  CRITERIA 


Criteria 

1 

2 

3 

4 

5 

Criteria  Weight 

i 

1 

7 

5 

2 

9 

0.448 

2 

1 

3 

0.053 

3 

5 

1 

4 

0.125 

4 

7 

5 

1 

9 

0.343 

5 


1 


0.031 
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FIGURE  4.  A  RECOMMENDED  APPROACH  FOR  MEASURING 
CAPABILITY  IMPACTS 

Risk  Intensity  if  the  System  does  not  Become  Open 
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Severe  Impact:  Enormous  shift  in  adversaries'  capability  or  technology  causing  major 
degradation  of  system  capability  (assigned  value:  twice  as  severe  as  moderate  risk  or  .60) 
Moderate  Impact:  Some  shift  in  adversaries  capability  or  technology  causing 
moderate  capability  degradation  in  near  future  (assigned  value:  three  times  as  severe 
as  negligible  risk  or  .30) 

Negligible  Impact:  Negligible  shift  in  adversaries’  capability  or  technology  causing 
negligible  capability  degradation  (assigned  value  =  .10) 


Using  the  above  preferences,  the  matrix  in  Table  2  was  constructed  for  the 
pair-wise  comparison  of  criteria  and  their  relative  weights.  A  number  of  AHP 
decision  support  software  (e.g.,  Expert  Choice,  simple  spreadsheet  algorithm, 
etc.)  are  available  to  bypass  the  tedious  calculations  and  quickly  find  the 
weights/priorities. 

Similarly,  pair-wise  comparison  matrices  for  four  candidate  legacy  systems 
(i.e.,  LHA  10,  LPD  12,  LCC  50,  and  LSD  70),  with  respect  to  each  criterion,  were 
established.  An  example  of  these  matrices  and  the  AHP  evaluation  criteria  is 
shown  in  Figure  5. 

Table  3  shows  the  final  assigned  weight  for  the  four  legacy  candidates. 


TABLE  3.  THE  OVERALL  SYSTEM  WEIGHTS 

Legacy  Systems 

Weight 

LHA  10  (System  A) 

0.140 

LPD  12  (System  B) 

0.114 

LCC  50  (System  C) 

0.262 

LSD  70  (System  D) 

0.484 
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FIGURE  5.  SETTING  LEGACY  SYSTEMS  PRIORITIES  USING  AHP 
EVALUATION  CRITERIA 
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LHA  10  (System  A) 

0.140 
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5 

Moderately  to  strongly  preferred 
Strongly  preferred 

LPD  12  (System  B) 

0.114 

6 

7 

Strongly  to  very  strongly  preferred 
Very  strongly  preferred 

LCC  50  (System  C) 

0.262 

8 

9 

Very  strongly  to  extremely  preferred 
Extremely  preferred 

LSD  70  (System  D) 

0.484 

As  mentioned  earlier,  it  is  desired  that  the  organizational  resources  be 
allocated  among  different  legacy  systems  in  direct  proportion  to  the  weights 
assigned  to  them  by  AHP.  Thus,  the  following  relationships  should  hold  true: 

Xa  =  ^  =^c_  =  Xg 
0.140  0.114  0.262  0.484 


The  following  equations  are  derived  from  the  above  set  of  ratios: 


0.114 

xA  - 

0.140 

xB 

=  0 

0.262 

xA  - 

0.140 

xc 

=  0 

0.484 

xA  - 

0.140 

xD 

=  0 

0.262 

xB  - 

0.114 

xc 

=  0 

0.484 

xB  - 

0.114 

xD 

=  0 

0.484 

xc  - 

0.262 

xD 

=  0 

These  equations  will  be  used  as  constraints  in  GP  model  formulation.  Using 
the  criteria  weights  shown  in  Table  2,  we  will  now  formulate  a  GP  model  to 
allocate  resources  among  the  migration  candidates.  As  discussed  earlier,  the 
priorities  among  the  criteria  will  be  used  as  penalty  weights  in  the  objective 
function  of  the  GP  model.  Table  4  shows  the  parameters  needed  for  formulation 
of  the  GP  model. 
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TABLE  4.  GP  CONSTRAINTS  AND  PARAMETERS 


Legacy 

System 

1 

Criteria 

2  3 

4 

5 

Legacy  System 
Weight 

LHA  10 

$3,000,000 

20 

15% 

20% 

15 

0.140 

LPD  12 

$1,750,000 

40 

10% 

25% 

30 

0.114 

LCC  50 

$6,000,000 

35 

5% 

10% 

20 

0.262 

LSD  70 

$8,000,000 

25 

5% 

15% 

35 

0.484 

Target  Level 

$12,000,000 

100 

20% 

60% 

80 

1.00 

Criteria:  1.  Improve  Cost  Effectiveness,  2.  Reduction  in  Obsolescence  Risk,  3.  Improve 
System  User  Satisfaction,  4.  Improve  System  Capability,  5.  Enhance  System  of  Systems 
Integration 


The  GP  model  for  this  problem  is  as  follows: 


Minimize:  0.448  N,  +  0.053  N2  +  0.125  N3  +  0.125  P3  +  0.343  N4  +  0.031  N5  + 
0.031  P5 
Subject  to: 

3  XA  +  1.75  XB  +  6  Xc  +  8  XD  +  N,  -  P,  =  12 

20  XA  +  40  XB  +  35  Xc  +  25  XD  +  N2  -  P2  =  100 

15  XA  +  10  XB  +  5  Xc  +  5  XD  +  N3  -  P3  =  20 

20  XA  +  25  XB  +  10  Xc  +  15  XD  +  N4  -  P4  =  60 

15  XA  +  30  XB  +  20  Xc  +  35  XD  +  N5  -  Ps  =  80 

0.114  XA  -  0.140  XB  =  0 

0.262  XA  -  0.140  Xc  =  0 

0.484  XA  -  0.140  XD  =  0 

0.262  XB  -  0.114  Xc  =  0 

0.484  XB  -  0.114  XD  =  0 

0.484  Xc  -  0.262  XD  =  0 

End. 

XA,  XB,  Xc,  and  XD  represent  the  contribution  level  of  each  legacy  system  to 
the  five  goals  specified  earlier.  Solving  the  above  GP  problem,  the  final  solution 
is  as  follows: 


XA  =  0.540889;  XB  =  0.440438;  Xc  =  1.012234;  XD  =  1.869929 
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This  solution  specifies  that  the  organizational  resources  should  be  allocated 
among  the  four  legacy  systems  based  on  the  following  percentages: 

LHA  10:  [0.540889/(0.540889  +  0.440438  +  1.012234  +  1.869929)]*  100  =  14  % 
LPD  12:  [0.440438/(0.540889  +  0.440438  +  1.012234  +  1.869929)]*  100  =  12  % 
LCC  50:  [1.012234/(0.540889  +  0.440438  +  1.012234  +  1.869929)]*  100  =  26  % 
LSD  70:  [1.869929/(0.540889  +  0.440438  +  1.012234  +  1.869929)]*  100  =  48  % 

Therefore,  legacy  systems  LHA  10,  LPD  12,  LCC  50,  and  LSD  70  should 
receive  14  percent,  12  percent,  26  percent,  and  48  percent  of  the  total  available 
resources  (funds),  respectively.  It  should  be  noted  that  legacy  systems  LCC  50 
and  LSD  70  receive  about  75  percent  of  the  funds.  This  high  resource  allocation 
ratio  is  commensurate  with  the  relatively  larger  contribution  of  these  legacy 
systems  towards  achievement  of  the  goals  and  objectives  specified  by  the  IPPT 
decision  makers. 

Besides  proper  allocation  of  organizational  resources,  the  proposed 
methodology  will  also  facilitate  and  expedite  the  migration  decision-making 
process.  This  latter  feature  of  the  proposed  model  (i.e.,  enabling  early  migration 
to  open  systems)  should  not  be  taken  lightly.  As  depicted  in  Figure  3,  failing 
to  migrate  in  a  timely  manner  can  result  in  degraded  system  capability  and 
effectiveness.  As  shown  in  Figure  6,  system  capability  and  effectiveness  is 
maintained  by  early  detection/prediction  of  capability  degradation  and  prompt 
migration  to  an  open  system  architecture. 


FIGURE  6.  EARLY  VS.  LATE  MIGRATION  STRATEGY 
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Advantages  and  Limitations  of  the  Proposed  Method 

Six  distinct  advantages  may  be  derived  from  the  proposed  methodology 
outlined  in  this  article: 

•  The  methodology  works  as  a  decision  support  system  that 
integrates  well-established  and  widely  used  concepts  (i.e., 

MOSA,  IPPT,  Delphi)  and  mathematical  models  (i.e.,  AHP,  GP). 

•  The  methodology  is  a  multi-criteria  resource  allocation  tool, 
which  incorporates  multiple  conflicting  goals  and  provides  a 
systematic  approach  to  establish  priority  among  various  criteria 
and  competing  alternatives. 

•  The  application  of  the  methodology  minimizes  groupthink, 
facilitates  brainstorming,  and  enables  reaching  consensus. 

•  The  methodology  enables  systems  engineers  and  architects  to 
develop  an  open  architecture  for  the  selected  migration  candidates. 

•  The  methodology  ensures  lower  total  cost  of  ownership; 
continued  system  effectiveness  and  capability;  and  extended 
useful  life  for  subsystems,  systems,  family  of  systems,  and  system 
of  systems. 

•  By  applying  the  proposed  methodology,  the  program  managers 
can  more  easily  integrate  systems  in  a  family  or  system  of 
systems  and  continually  leverage  the  investments  made  in 
commercial  products,  practices,  and  technologies. 

Some  limitations  of  the  proposed  method  are  the  tedious  and  perhaps  time- 
consuming  IPPT  process  for  establishing  objectives,  and  determining  the  entries 
for  numerous  pair-wise  comparison  matrices.  The  use  of  complex  mathematical 
formulas  may  be  another  drawback  of  the  model  although  powerful  software 
exists  for  AHP  and  GP  computations. 
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Conclusions 

This  article  proposed  an  integrative  method  for  assessing  the 
appropriateness  of  migrating  closed  legacy  systems  into  affordable  and 
adaptable  open  systems,  and  developing  open  architectures  for  the  selected 
candidates.  The  method  employed  the  MOSA  concept,  applied  an  Integrated 
Product  and  Process  Team  approach,  and  used  Analytic  Hierarchy  Process 
and  Goal  Programming  models  as  an  integrated  multi-criteria  decision¬ 
making  model.  The  methodology  capitalized  on  AHP  and  GP  benefits  such 
as  objectivity,  consideration  of  tangible  and  intangible  factors  in  decision 
making,  and  simultaneous  incorporation  of  multiple  conflicting  objectives  in 
the  decision-making  process.  The  methodology  also  took  advantage  of  open 
systems  benefits.  The  benefits  of  open  systems— such  as  adaptive  modular 
architecture  and  increased  portability  and  interoperability— can  significantly 
enhance  an  organization’s  core  competencies  by  reducing  the  total  cost  of 
system  ownership,  increasing  long-term  viability,  and  shortening  the  length  of 
development  cycle  time  for  systems  and  applications. 
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TECHNOLOGY  CORNER: 

GAMES  FOR  GOOD- 
HOW  DAU  IS  USING 
GAMES  TO  ENHANCE 
LEARNING 


\  Alicia  Sanchez 

The  use  of  games  has  become  a  popular  initiative  in  many 
learning  organizations.  The  Defense  Acquisition  Univer¬ 
sity,  by  targeting  enhanced  outcomes  for  learners  and 
using  innovative,  multiple  approaches  to  develop  games 
and  simulations  that  are  engineered  to  yield  performance- 
oriented  outcomes,  has  created  a  unique  opportunity  to 
reach  an  evolving  workforce  on  multiple  levels.  Through  the 
use  of  story  and  interaction,  students  gain  a  better  under¬ 
standing  of  the  dynamic  application  of  course  content, 
while  fostering  motivation  to  learn  and  increasing  perceived 
relevance  of  the  instruction.  This  article  covers  the  use  of 
games  and  simulations  in  three  different  initiatives:  Games 
in  Curriculum,  Games  in  Continuous  Learning  Modules,  and 
Mini-Games— each  of  which  was  created  with  the  end  result 
of  learning  in  mind. 
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Background 

As  the  Defense  Acquisition  University  (DAU)  prepares  for  a  younger 
workforce  and  the  increased  performance  capabilities  that  will  be  expected 
of  that  workforce,  it  has  simultaneously  transitioned  its  curricula  from  a  more 
traditional,  educationally  focused  organization  to  a  true  training  organization. 
Within  DAU,  the  necessity  for  preparing  students  to  perform  at  higher  levels 
has  generated  an  increased  emphasis  on  providing  students  not  just  with  the 
information  they  need  to  do  their  jobs,  but  also  the  skills  necessary  to  accomplish 
the  unique  challenges  they  will  face  upon  arrival  in  their  workplaces. 

This  combination  of  unique  needs  has  caused  DAU  to  innovate  and  transform 
a  more  traditional  educational  program  into  one  that  accepts  the  increased 
use  of  online  learning  and  reductions  in  classroom  “seat  time,”  presenting  an 
opportunity  to  rebuild  the  university’s  classroom  lecture  environment  into  one 
that  more  closely  aligns  with  a  hands-on  apprenticeship.  This  rebuilding  is 
made  possible  through  the  caliber  of  education  that  DAU  provides,  particularly 
in  its  incorporation  of  a  games  and  simulations-based  initiative,  which  allows 
students  to  experience  workplace  events  that  they  will  face  during  their 
classroom  and  online  learning. 

While  simulations  have  traditionally  been  used  to  provide  people  with 
experiences  that  they  might  otherwise  not  have  because  they  are  either  too 
expensive,  too  dangerous,  or  happen  too  infrequently,  simulations  are  more 
frequently  being  used  to  address  performance-oriented  learning  objectives  in 
educational  and  training  environments.  Most  simulations,  however,  are  linear 
and  are  only  used  once.  To  increase  the  benefits  of  using  simulations,  DAU 
incorporated  gaming  characteristics  into  its  curricula  because  of  their  proven 
capacity  to  increase  a  student’s  motivation  to  interact  with  the  course  content 
in  meaningful  and  targeted  ways.  This  is  accomplished  through  the  inclusion 
of  high-level  storylines  to  serve  as  a  unifying  scenario  for  courses,  and  game 
themes  in  order  to  support  the  use  of  increased  interactivity.  Specifically,  using 
storylines  makes  it  possible  to  present  content  that  is  more  relevant  to  learners. 
And  relevance  is  a  key  component  to  increasing  the  “perceived  value”  of  the 
learning  experience  to  the  learner.  When  motivated  learners  are  presented  with 
information  in  the  appropriate  context,  gaming  also  facilitates  the  ability  of 
learners  to  integrate  that  information  into  the  mental  models  where  individuals 
retain  and  recall  their  workplace  experiences.  Therefore,  providing  the  context 
of  the  information  being  presented  can  increase  a  student’s  ability  to  understand 
why  and  when  that  information  can  and  should  be  used.  This  is  also  theorized  to 
lead  to  increased  ability  for  a  student  to  transfer  that  learning  experience  into 
their  everyday  workplace  experiences. 

In  short,  the  use  of  game  characteristics  can  enhance  the  learners’  ability 
to  make  meaningful  connections,  retain  and  transfer  knowledge,  and  motivate 
them  to  interact  with  course  content  by  providing  experiences  when  combined 
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with  simulations.  When  combined,  games  and  simulations  have  the  ability 
to  provide  the  learner  with  experiential  learning  opportunities  that  focus  on: 

•  Appropriate  Context— The  ability  to  allow  learners  to  use  the 
content  in  scenarios  or  situations  that  are  representative  of  how 
the  content  would  be  used  in  the  real  world. 

•  Cognitive  Fidelity— The  alignment  of  the  content  with  processes 
that  are  representative  of  how  the  information  would  be  used. 

•  Varied  Situations  and  Replay  Opportunities— The  ability  to 
provide  multiple  practice  opportunities  for  content  use  based  on 
scenarios  and  situations  that  are  different  in  key  ways  in  order  to 
expose  the  students  to  a  variety  of  experiences  with  the  content; 
and  the  ability  for  a  learner  to  use  these  tools  multiple  times  in 
order  to  try  new  strategies  for  how  the  content  could  be  used. 

•  Self-Diagnosis— The  ability  for  learners  to  gain  an  understanding 
of  their  strengths  and  weaknesses  in  order  to  provide  them  with 
the  information  they  need  to  self-monitor  their  learning  process. 

•  Scaffolding— The  ability  for  information  to  become  increasingly 
complex  as  a  learner  evolves,  with  previous  information  being  built 
upon  in  order  to  provide  a  gradual  increase  in  their  understanding. 

Through  these  game  characteristics,  it  is  possible  to  provide  students  with 
the  necessary  motivation  and  relevance  (Figure  1)  to  internalize  course  content 

FIGURE  1.  RELATIONSHIPS  BETWEEN  GAME  CHARACTERISTICS 
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and  transfer  the  knowledge  gained— complemented  by  their  own  personal 
abilities— to  performance-oriented  outcomes. 

Following  much  time  spent  considering  where  and  when  games  could  make 
a  difference,  implementation  of  three  separate  types  of  games  emerged  to 
address  the  specific  needs  of  DAU’s  diverse  student  population  and  curricula. 
These  three  categories  vary  in  level  of  specificity  of  games  and  are  detailed  here. 


GAMES  IN  CURRICULUM 

The  first  and  most  specific  category,  Games  in  Curriculum,  focuses  on 
games  that  are  aligned  highly  enough  with  a  course’s  learning  objectives  and 
context  to  warrant  a  game  being  placed  in  a  course.  Since  acquisition  is  a  very 
specific  process,  often  these  types  of  games  are  naturally  going  to  be  custom 
developments.  In  these  types  of  games,  it  is  important  to  consider  the  aspects 
of  the  algorithm  that  predict  the  meaningful  use  of  a  game  in  order  to  establish 
the  highest  probability  of  enhancing  performance.  One  example  of  this  type  of 
game  lies  in  the  use  of  a  series  of  games  within  the  Business,  Cost  Estimating, 
and  Financial  Management  curriculum.  Specifically,  a  low-level  course  that  serves 
as  a  required  course  for  all  students  in  the  career  field  was  selected  to  include  a 
series  of  related  games.  While  the  course  selected  represented  a  high-performing 
course,  a  content  analysis  of  the  course  indicated  that  this  course  transmitted 
primarily  conceptual  knowledge,  often  including  a  heavy  emphasis  on  vocabulary 
memorization  while  providing  case  information  of  little  use  or  context.  It  was 
hypothesized  that  by  including  games  at  the  end  of  each  of  the  online  courses— 
eight  modules  that  provided  situations  in  which  the  information  being  memorized 
could  be  used— students  would  find  more  relevance  in  the  information  being 
presented,  and  therefore  would  be  motivated  to  retain  the  information. 

The  game,  named  Rat  Race  (Figure  2),  centers  around  a  story  of  a  rat  who, 
after  having  lived  in  the  Pentagon  for  many  years,  has  mastered  the  art  of  business 
financial  management  by  listening,  befriending,  and  assisting  some  of  the  world’s 
greatest  minds  in  the  acquisition  process.  Under  this  frame,  players  are  able  to 
practice  the  lessons  learned  within  the  context  of  their  real-world  application,  but 
using  a  fantastical  character  that  is  both  endearing  and  engaging. 


GAMES  IN  CONTINUOUS  LEARNING  MODULES 

Continuous  Learning  Modules  (CLM)  at  DAU  are  very  specific  2-  to  4-hour 
online  learning  modules  that  supplement  the  core  curriculum.  They  often  target 
specific  career-centered  information  related  to  process-oriented  performance 
characteristics.  In  the  case  of  one  CLM  in  particular— Procurement  Fraud  Indicators 
(PFI)— the  content  of  the  module  deals  with  the  ability  to  identify  fraud  when  it 
takes  place  in  the  workplace.  Like  many  judgment-sensitive  areas,  the  nature  of 
this  content  is  shaded  in  gray  area,  innuendo,  and  non-yielding  protocol.  The 
continuum  gap  between  novices  and  experts  in  these  sorts  of  content  areas  is 
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FIGURE  2.  REPRESENTATION  OF  THE  BUSINESS,  COST,  AND 
FINANCE  GAME 


often  filled  with  personal  experiences  and  first-hand  accounts.  In  the  absence 
of  those,  the  use  of  an  experiential  game  can  serve  to  provide  students  with  the 
varied  situations  and  opportunities  to  practice  in  a  safe  environment. 

The  PFI  capstone  game  is,  at  heart,  an  adventure  game,  not  unlike  traditional 
point-and-click  adventure  games  designed  for  the  personal  computer.  The  game 
is  divided  into  several  different  scenarios,  each  focusing  on  a  specific  character 
that  is  suspected  of  committing  fraud.  Each  scenario  is  divided  into  two  phases. 

Phase  I.  The  “Hidden  Object  Phase”— an  exploration  mode  where  the  player 
visits  two  scenes  to  collect  clues  and  piece  together  information  about  the 
fraud  being  investigated.  The  scenes  differ  between  each  scenario,  and  range 
from  the  mundane  (a  home  office)  to  the  exotic  (a  Navy  vessel). 

Phase  2.  The  “Interview  Phase”— the  point  in  the  game  when  the  player 
moves  to  the  interrogation  room,  where  the  suspect  is  questioned  about  the 
situation  to  gather  more  information.  In  the  Interview  Phase,  three  theories  are 
presented,  each  using  the  same  clues  to  draw  different  conclusions.  The  player 
can  investigate  any  or  all  of  the  theories,  drawing  his  or  her  own  conclusions 
from  the  information  presented. 

At  the  end  of  each  line  of  questioning,  the  user  can  attempt  to  identify 
the  fraud  that  occurred  and  the  indicators  that  suggest  it  (encapsulated  by  the 
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three  theories).  Finding  the  correct  fraud  with  the  correct  reasoning  results  in 
the  player  “winning”  the  scenario;  otherwise,  the  player  can  try  other  theories 
until  successful. 


FIGURE  3.  REPRESENTATION  OF  THE  PROCUREMENT  FRAUD 
INDICATORS  GAME 


MINI-GAMES 

Mini-games— simple  downloadable  games  that  are  commonly  found  in 
conventional  Web-based  training  courses— should  no  longer  be  considered  as 
nothing  more  than  a  distraction,  breaking  up  the  content  from  the  inevitable  test 
that  will  be  presented  on  the  next  page.  By  applying  new  design  patterns,  mini¬ 
games  have  come  into  their  own  as  a  legitimate  form  of  training  and  education. 
DAU  is  incorporating  mini-games  into  a  Web  repository  that  allows  students  to 
brush  up  on  their  core  topical  knowledge  of  acquisition-related  competencies. 

Mini-games  are  usually  small  games  that  are  easy  to  learn,  but  hard  to 
master.  Anyone  can  play  “Tetris,”  but  it  is  difficult  to  play  Tetris  well.  While 
conventional  games  might  take  days  or  weeks  to  play,  mini-games  are  typically 
played  for  less  than  an  hour.  Educational  mini-games  follow  the  same  philosophy 
and  contain  a  single  learning  objective.  Furthermore,  the  design  of  mini-games 
has  matured  from  simple  matching  games  and  quizzes,  to  real  and  meaningful 
interaction  with  training  concepts  as  will  be  demonstrated.  These  mini-games 
are  used  as  homework  assignments,  remediation,  pre-course  materials,  or  just 
as  stand-alone,  self-motivated  training  by  the  AT&L  workforce. 

One  such  game  currently  in  production  was  designed  to  introduce  students 
to  the  seven  tools  of  Continuous  Process  Improvement  (CPI).  This  game  focuses 
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on  a  student's  ability  to  understand  when  and  why  the  tools  are  appropriate 
for  use,  using  a  fantasy-based  alien  production  line  (Figure  3).  The  story  that 
surrounds  this  game  introduces  an  alien  army  heading  towards  earth.  The 
student  does  have  a  weapon  that  can  defeat  them,  but  only  a  limited  amount  of 
time  to  finish  production  of  enough  weapons  to  defeat  the  aliens.  At  the  current 
production  rate,  there  will  not  be  enough  weapons  to  save  earth.  Using  proper 
CPI  methods,  players  have  to  improve  the  process  currently  in  place  to  increase 
production  and  save  mankind. 


Conclusions 

Through  insertion  of  the  three  types  of  games  and  simulations  discussed 
in  this  article  into  its  course  content,  DAU  will  have  an  opportunity  to  measure 
the  success  of  games  and  simulations  and  their  utility  in  learning.  The  insertion 
of  Games  in  Curriculum,  Games  in  Continuous  Learning  Modules,  and  Mini- 
Games  into  DAU  course  content,  as  well  as  several  other  projects  currently  in 
development,  will  pave  the  way  for  the  university’s  ultimate  transition  to  a  more 
hands-on,  apprenticeship-type  learning  environment,  increased  motivation, 
and  increased  relevance  for  students  through  interactivity  and  experiential 
learning  tools.  Together,  these  will  help  engender  a  culture  of  performance- 
oriented  learning,  culminating  in  an  across-the-board  higher  level  of  on-the-job 
performance  excellence. 
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